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^grades than* those that had been studied eariier. Stepwise ' discriminant 
analysis was. used to determine ^ how.. well the quantitative variables derived 
from the sJhort-Qut content analysis of the narratlj^'^text could classify 

* each Individual evaluated into correct criterion groupi, ^Jiis investigation 
was intended tb sh<3w if the rational condensation shoyt-cut indexing proce- 
dure 'is generalizable (1)^ from AT's, BT's, OS's, artd-^'s to additional , 
occupational specialties and (2) from Pay Grade^EV to Pay Grades E5 and E6 

Using a subsample of the E5-£6 data base, a reliability study vas con- 
, ducted 'In order to be*certain thelt consistency amortg several in^exers can 
be taught and^achiev^ in their interpretation" and application of the ra- 
tional condjensation short-cut indexing methi5d.' In this second task, the 
level of agreement between each of the four reliability indexers attd the ex- 
perienced indexer who trained^hem was determined by the same statistical 
procedures (k^ppa, weighted kappTa, apd product-moment correlati.t)ti) used Ivy 
^ two earlier reliability studies 'in order that comparisons could he made' 
among, the* three reliability stlidies of the magnitude of agreement 'achieve^. 
This investigation was intended to lay the foundation for a training curricu- 
lum, that may bp* used in the future to train Navy ^nd civilian -operational 
personnel in the applicat^an of the ^content analysis methodology. 

The results of the first ^wo rey.ability studies suggested the -possibili- 
ty that it may -be as important* to consider the issue of internal consistency 
for a single indexer as ,to Measure, the level of agreement that can he a- 
ehieved among several Indexers. It seems reasonable to assume that although 
there ma,y be slight differences between two indexers in how they apply a »( 
pa-cticular indexiii^g procedure, a more important -Gons^ideration is that they^ 
consistently ub§ their own individualized interpifetation of'the indexing 
rules and conventions. One then'^might jexpect that regardless of which -in-^ 
dividualized interpretation ^was used ' to IndfeJc a particular data base^ a 
similar^le^vel of classification agreement witli the criterion of on-job per- 
formance cduld, be^'achievedfi^ This is an important area to study because the 
findings may point to the necessity to u^e only one indexer. for a pe^ticular 
data base^ if optjj^um extraction of differentiating inf ormatiqn . is to be ob- 

' tained. 'A third ^ask vjas unflertaken in order to shed som^ light-on this * 
issue by^havirig a second indexet Independently reindex two earlier data sam- 
ples using the original lengthy indexing procedure. Thus^ an exact^repUca- 
tion of- thp indexing performed'Tby the first , indexer in her^^ontent analysis 
of these*^ initial sartpies yas cat^ied ou^t independent:ly . iThe accuracy of ^ 
classification ijitp correct criterion group achieved, by each of these two 
indexers was.^compai^ed in order to determine" if both indexers workings se'pa- , 

- rately with their own individualized . interpretation of"^ the .indexing rules 
and conventions could achieve comparable classif icattbn resultp. In' addi- 
tion, the-rpossibllity that two indexers sharing the indexing of the same * 
data base can achieve as good classification results' as either indexer in- 
dexing the entire jdata base alone was ex^ilored. 

In the foiyth- task, a careful comparisoh of' the indexing, coding, key- 
punching, -an/ subsequent computer, ^rocessilag time required to apply the 
rational compensation short-cut indexing procedure ^ and' thfe original lengthy 
indexing, procedure to smarll /subsample of the ES-EB data base waS made. 
This comparison provides data needed for assessing' the economic feasibility 
of adding inf ormati^on fextractfed from narrative ^comments into a composite 
score for predicting an- enlisted man's potential for as'feuming^ the mana^gerial 
responsibilities of the rjiextP higher pay grade. V 
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training tn^riual for the rational condeji^atibn shot^t:^^^^^^^ . 

^ reproduced in Appendix A also Is her x^drk. Het conj-lnn^ con^ 
tributions to thf 'successful » development and doctiTtientatioh. d£ Qptttent analytic 

* techniques for analyzing narr^l^e pr^for^ evaluations; a^ arid 
deserves special mention* ' ' "■ ■^^f ' 'V--:'*" \ ^ ' 

'v v; Jonnie'Handiey was the second in^^r ^^ndependentljg^h^idex tVo earlier 
samples of narrative performance evaluatlbn^T^or s^^br en^ti^ in 
^ Pajr jlrade EZ. She also served as JtelMiility Indexed ^^^^^^ as performing 

alO: of thie statistical calculations f bl%ie tellability studj^^^f' ^i^^ 
Condensation short-cut Indexing method. Hef^dicated and^iSc^ 
performance of her part of this research is acknowledged with gratitude. 

' . . " * * ^ ' ^* Ramsejr-Klee, Ph.D. 

' y ^ * ; . ^ Pt;incipal Investigator 



A jgrear deal,of diffei^entiatte^^ 
p0j*ic)rmatt,ce . Evaluation f ornfii that cquld ^nbstaistial iii selecting P 

sonn^l for advancement, duty assignment, trali^ping^ or quality ^et^Mon is not 
being j^loited in any aysrtematic ttan^^ eagy, 
analyslav ^Tlife??^ aimed at^ 

devglopiiig and #lf initig contetit sttiai^rfcic te the ^ 

differentiatirig information in 'narrative pe^ ei^^tiat±oi\& f o£ ^enlisted * 

personnei.s tn the fourth study being ifeport^ webe^^ 
takeu: (1) to c^osd validate thV $upetior aticdt^dx^ t:echniqi;e; devel-^ 

oped earliet— —the rational condett$ation metAi^c^^ spe- 
cialtles and on other p^y grades th^ Jihose- th|i^^ previously , 

(2) to conduct 'a third reliability 3 tudy to' >e q^ffcain ^at^cbn^ among 
several indexers can be taught and achieved in their interprets^tion and appli- 
catlbn of the rational condensation sho;^t~cu£ indexing method^, (3) to Teliidex 
two earlier samples ^f j^krratiye coTO^s^^u^ 

^prqcedure and a -s^ecpnd itidexer to .winpare the accuracy of the two indexers in 
Nilassif yittg senior enlisted persohhel itito correct criterion groups based! on 
job pef formance marks In order -to deterinine if Ijoth indexers working sepatate^ 
ly with their own inaivldualized interpretation of the indexing rules andj-con- 
ventioiis could achieve doniparable classification results and if two index^r3 
sharing the indexing -of the same data base ^can achieve as good classification 
results as either indexer indexing the entire data baser alone^^ and ^4) to com^ 
pare the time required to apply the rational condensation short--cut indexing 
procedure and the original lengthy Indexing proV^uire to a small sample of 
Evaluation Reports, . - ^ . 

Cross Validation of the Rational Condensation Short'-Cut 
' ' ' Tn^exixicf Procedure on the E5-E6 Fleet Trial Data 

The next step, in this research was to cross validate the better of two 
short-cut indexitig techniques used in classifying three E7 data bases studied 
earlier on other occupational ratitj^gs and on other pay grades* A set of 
useable fleet trial d^ta, including criterion dafa, were available at the Navy 
Personnel Research and Development Center (NPBDC) that had been generated An 
the" process of testing a number of ekperimental f<%rtos for measuring, on-jbb 
performance for Pay Grades E5 and E6 in seven occupatldhal spedialties. A 
30ff-case sample of Evaluation Reports representing six of the seven Occupa- 
tional speci^ties itt the ES-E6 fleet trial data baae was selected, including 
Aviation Machinists Mate (AD) Damage Controlman (DC) , Electronics Technician 
(ET) ^ Personnelman (PN) , Radliotiatt (RM) , . and Storekeeper (SK)* A set of 23 
quantitative variables was derived from the, rational condensation short-cut 
indexing form used in the content analysis of the E5-E6 fleet trial narrative 
performance evaluations* Profiles or vectors pf these 23^yarlables were , 
constructed for all" of the Evaluation Reports contalijii^d' 1^ the 300-case fleet 
trial sample, separate profiles bein^ Compiled for the Evaluation Section and 
the Juetiflcation Section of each Evaluation Report # Theee profiles then were 
sutifjected to stepwise discriiainant analy^s* . ' > / 



In th^ analysis, pf jaarrative comments contained in the Evaluation Section 
4ni the Justification Section for ea^ih of the six occupational specialties ; . 
retresejited in th^ 3Q0~c^se ^-1^6 fleet tri.al sample, apt>roximately^half or 
more of the cases^ were classifxed/into' correct "criterion groyp at the very 
first step in tKe st:epwi3e-^isfiriminant^analysis process^ After the first, 
step, classification peifformance continued, to climb slowly but asymptotically. 
However, perfect classification performance was not achieved. for any\?f-,the 
six' occupational specialties on either section of narrative comments, although, 
classification accuracy" i;anged 'from 67 to 50 piercent. In earlier content * 
analysis st[;tidies at the E7 pay 'grade level^ better clas'sificatjlon performance ^ 
>was achieved than in tliis sample. V However,- at the lower ^ay grades there 
typically are fewer words in the narrative te:^t of justification comments. If 
fewer words also indicate less- substantive ^ontent, then there is less poten- » 
tidrlly .differentiating information in the Narrative comments written by evaLur 
ators f or enlisted p'ersonnel in Pay.Grades E5 and E6. ^ 

Sevejral re^sbns may help explain these results. First, in the lower pay"*' 
grades job duties which are leSs demanding, more routine, 'and not so manage- 
rial in nature may produce less critical observational data and narrative 
comments that lack the necessary substance to differentiate between good and 
superlative first and second class petty officers, evidenced by silbstatitiapy 
fewer words in the narrative text- for Justification comments at the^lpWer pay ^ 

^grades. Second, the^ sample sizes in the six occupational specialties . repre- 
sented in the E5-E6 fl^et tirfal! sample are quite small in comparison to those 

^"-studied earlier. Third, the six occupational specialties represented in thd ^ 
E5-E6 fleet trial sample may constitute jpb categories that afford l^ss dif- 
ferentiating performance *data than the four occupational specialtiips repre- ,^ 
sented in the E7 s^ples -studied earlier. ' . ^ 

Classification results, oti the Evaluation Section were* comparable in t^e^ . 
.two samples for the one occupational specialty in cpmmon in the E5-E6 and E7 

samples— Radioman ^ (RM) . However, better classification accuracy was achieved 
-on the Justification Section for the E7 R^'st, probably ^ecause' of the lengthier 

narrative text in j,ustif Ication comments at the higher pay grade level. ; 

As in earlier research, better classification usinfe the stepwilse cdisdrim- 
inanV analysis procedure was aChieve^jTon the E5-E6 fleet trial sampl'e b]j \,' ^ 
analyzing each .occupational specialty separately rather than by 'combining all 
of the occupational specialties' represented In. sample. ^ - , . \ 

Most of- the misqlassification^ jnade by thfe s^tepwlse discriminant Analysis 
procedure, as 'judged by the WitericTn of op-^job performance, wete in the 
direction of classifying "an indi\ridual into a lower criterion ■ group than the 
one to which he aat4aally belonged.. Since the -criterion of on-job .performance 
is inip^rfect, the stepwise discriminant analysis procedure appear^ to sift out 
\he individuals Who might have been assigned to a higher criterion group be- 
cau^^ of -inflated marks.. Application of thdbe statistical tecbpique wc^uld help 
to narrow thd field of candidates, for advancemeijt , duty asstgnmenr, training, 
or .quality retention to only those individuals-- potentially the most qualified. 

"An examination of the variables se^^cted by the a*:epwise discriminant 
analysis program for the first 15 steps iti- each of rhe computer runs reveals a 
pairticularly interesting set of results since mos^ of the classification 
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problem Is solved fti Che early steps of the stepwise discriminant analysis. 
Jfor both/ sections of narrative cqtliiaetits all 23 quantitative variables derived 
from the rational condensation $hort-cut indexing procedure were selected ^ 
within the first 15 steps by at least one occupational specialty suggesting 
that all variables used in the stepwise .discriminant analysis have something 
to contr;ibute to solving this classification problem. A particularly differ- 
entiating variable is Total Number of Index Terms Used. .In earl;J.er content ^ 
analysis studies* at the E7 pay grade level, without exception the first- vari^ 
able selected for the Justification Section was' Total Number of Index Termsf 
Used, reflecting the variety^f specific areas of ^n individual's performance 
that the evaluatoir chose to cotmnent on, and Indicating that the range of 
skills and' abilities » that a chief peTtty officer manifests is a key factor in ^ 
his superior performance. At the E5-E& pay grade levels. Total Number"^ of 
Index Terms Used is a key discriminating variable in both the Evaluation 
Section' and. the Justification Section/ j » . 

The number and type of adjectives and adverbs that an evalnator uses t<f 
describe the performance of the individual (as reflected in the. various/ 
weights assigmed tO' index terms) do not appear to contribute very much to 
d if fferentiatlng among superlative first and sedohd Claas petty ojcf leers and 
their slightly, less qualified peers,, whereas at the E7 pay grade l^vet (chief 
petty officers) these weights are impoiftant discriminators. There appears to 
be less critical observational data available at the lower P^Y grades upon 
^which to base a performance evaluation as reflected in shorter narrative 
■QoipDients on the Justification Section.. , ' 

When both sections of narrative commi^nts are considered together, the 
/more discriminating vaifiables seem to te Total Number of Index Terms Used, 
Cooperation and Responsiveness , Endurance 'and Motivation, ^productivity and 
Achievement, Tnteliectual. Functioning, Recognition, and Total Number of Words 
in Text* : At Pay Grades E5 and E6, it seems reasonable tTiatJa petty officer's 
cooperative and responsive spirit, his endurance and motivation, his pro- 
ductivity and achievement, his. level of intellectual functioning, and, the - 
amount of redognition he rec^eives for his on-job performance would be more 
significant variables than leadership^ and management qualities that will be 
brought into play later in his career as he- advances through the pay girade^ 
structure from junior to senior enlisted petty officer * ' 

: . - , • ■ . • "-'^ ^. . ■; ; ■ / . . 

^ • T^eliablHty Stu^y of the 'Ratioxtal Condensation ^ * 

I V 5hort-;^ut Indexing Procedure 

A new training manual was prepared to explain and illustrate the proper 
utiliization^of the rational' condensation ahort-cut indexing procedure. Four 
reliability indexers were trained by the experienced iiidexer over the course 
of six eesslons in the applioatidh of thip procedure using the training manu- 
^al. These four indexers then'indfependentiy indexed the narrative Comments 
contained in a newly selected subset of 48 Evaluaf;ion Reports taken from the 
E5^E6 fleet trial data base* Once again, the^valuation Section was. separated 
from the'JuatiflcatlQft Section, resulting in a group of 96 randomized pieces 

of narrative text minidocumes^S' that were -Jndexed independently *by each . 

reliability indexer. Their indexing deciliipns then were compared to those of 



the experienced indexer who trained th^m'^nd^whose decision-making processes 
they were trying to emulate. The same agreement statistics- tl>at were used in 
elie two- previous reliability studies (feappa, weighted "kappa , and product- 
HKMp^nt correlation) were computed in' order to determine- if the reliability 8f 
tjlfe rational condensation short-cut Indexing procedure is comparable tcr«-that 
<found iti' the two earlier, reliability studies "^f the originaf lengthy indexing 
procedure. The "value "oT the various agreement statistics ranged from .73 to • 
.90 in this third rfeliability study, excluding the, results for one' r'eliability 
indexer who suffered a whole host' of .personal lyroblems during the course of 
the study that directly affected her indexing performance. . ^ 

" Thus, consistency among several indexei>s in their interj)retiation and 
application of^ the' i^ational condensation short-cut indexing procedure to the 
narrative coiranen^ts contained Itt, performance evaluations for naVal^ enlisted 
personnel Is high and better than that achieved with the lengthy indexing 
proced^ice in two earlier reliability studies. In the. 197^2 and l973 reliabil- 
ity stupes of the lengthy indexing ptocedure, better agreement^i^th the' 
experienced Indexer was exhibited in the' selection of index tetms compaifeid to 

'the assignments of weights to these terms based on the modifying ad|ectiv6s and 
adverbs use«i by an ^Valuatoir. . In the 1974 reliability *study of the rational 
condensation short-cut indexing procedure , agreement with the expefleneed, 
indexer was approximately the same for the selection of Index tertns and the 
assigfliJ^nt^ of weights to these terms. In all three reliability studies, quite 
respectilpe levels of agreemfeift bietween 'the experienced Indexer and the' vari- 
ous Keliibillty indexj^s were achieved In only six training sessions, indi- 
cating that" Navy and civilian bperational- personnel also should be able to : 

, learn to cbnsistentiy apply the cdntent analytic techniques developed in this 
research project. : 

* Validation of the Original Lengthy Indexing . 

■:. Procedure by Means of a Second Indexer 

. " ■ *■ ■ " . 

Tbjg;' results of the first two re^abllity studies suggested' that It may be 
as .important to consider /the i&su& of Internal consistency for a .single in- 
dexer as to measure the level of agreement that can-be achieved among several 
indexers. . Although twtj indexexs may differ slightly in how they apply a 
particular Indexing procedure, it may be more impor^tant that_they consistently 
use their own individualized interpretation oT the indexing rtt\es and conven- 
tions. Regardless of the individualized InterpCetl^tion used to in^ex a par- 
ticular data base, a -similar level o^ classification, agreement with the cri- 
terion Of on-^b performance might be achieved. In orijer to shed some light . 
"on this issue,- a second indexer independently relndexed .the cross validation 
and generalization samples using the orlgindl lengthy Indexing procedure. The 
accuracy of classification into correct criterion groiiip achieved by each of 
thes6 two indexers was compared. . In addition, the possibility that two in- /; 
dexers sharing the indexing of the same data base can achieve as good classi- 
fication .results as either indexdr indexing the entire database alone was 
exploredi . v 

Regardless of which individualized interpretation" of fee indexing rules 
and conventions w'as used/ to index a particular data base of narrative perfor-. 
mance evaluatlori», a similar level' of classification agreement with the cri- 
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teribn of oii-job performance, was^achievefdw "furthermore, it can be concluded 
that two indexers sharing the indexing of the same data base can be expected ^ * 
to achieve as good classification results as either^ indescer indexing the 
entire 'data base alone. Therefore', there 'apnears to b§'n6 necessity to use- 
only one indexer for a particular data tasejln order to obtain optimum extfac- 
tl^on of differentiating information,, assumi^ that indexers ard well trained 
to begin wit;h and conscientious in applying the_indexing rules aftd conventions 
to the best of their ability. ' . *> ' ' • . 

, V / . ' 

' . : - ^ V ^ ^ ■ •% . ^ ^ - ; . . 

' 'Efficiency of tRe Rational Condensation ShQrt-Cut^ Indexing' 
Procedure Compared to iihe OrigJ.nal Lengthy. Indexing Procedure 

. Althouglj, the rational condensation* short-cut indexing procedure'in 
earlier studies did not achieve the classification ^ccuracx of the original, 
lengthy ind^xihg procedureT^t did adhieve, an acceptable level df classic 
fica'tion performance. It is much easdjer to appjy a^d apig^ars to be slightly 
more reliable \since there ^e fewer airlas of ambiguity, resulting in more 
consistent interpretJatiOn of the indexing ruJLes and .conventfbns. therefore, 
an important conaider'atipn *is hpw efficient each of these, two induing pro- 
cedures is for indexing a particular sample of Evaluation ^leports. This' kind 
of GOTt^parison c^n provide data needed for assessing the economic feasibility 
pi adding information .extracted from narrative comments Into ^composite score 
'for 'predicting ah enlisted man's potential fox assuming the managerial re- 
sponsibilities cte^ the next higher p^y grade. ' / * 

A subsample of* 12 Evaluation Reports tak^ frotn the E5--E6 fleet trial 
data base was selected. The time tequired to iapply the ^ rational condeh^tioti^ 
short-cut indexing procedure and -the original lengtby indexing, procedure \o ' 
this 12-case subeamgle was compared. Taken into account in the comparison was 
4:l^e time required bj^^both itidexing procedures (1)* to index' the narratiye^text 
for the Evaluation Section and the Justification Section," (2) to enter^ the 
resulting indexing decisions onto the^indexing forin, (3) to code the data 
recorded oh' tl)e Indexing forms onto IB^ coding sheets,^ (4) to keypunch the IBM 
coding "Shedtff,- and (5) to keytrerify the IBM coding^heets . Tha computer 
procesaing time required 4>y each of tbe'^'two indexing procedures also was ' 
'considered. • V • . 

In or^der to arrive at some estimate. q| the compar^^tiye costs of applying 
the twp indexing procedures to a typical lOO-case . sairfp:!^' of E5-E6 Evaluation 
Hepotfes, cost estimates were prepared based on the time required^per case to 
pejrform all of tfxe steps in the conterlt analysis of the narrative text up to\ 
thjg point of 'computer arilalysis. The cost -comparison suggested . that the ^ 
rational condensation short-cut indexing prpcedure can bfe applied for ''about 60 
percent the cost of applying the lengthy indexing procedure* tp the same 
corpus of narrative text. The difference in cost in favor of the short-cut 
indexing proc^ure for the computer analysis id insignificant; amounting to 
only two^r three dollars for a sajnple size of around 100 cases. Since little 
is* lost in classification acc\iracy by using the short-cut indexing procedure, 
•tl^e economic advantage of this indexing method opts in its favor. The only 
justification for using the longer, more coioj^lex indexing methodology might be 
in situations Where it waa expected that distrii^inat ion between criterion 
groups would be extremely diff icult to l^chleve. . ' 



SECTION 1. INTRODDCTtON AND BACKOROUND * , . 

^ .The purpoW of this pontrlnulng researcji Investigation has been to comple- 
meat the Navy Personnel Resea;xh and Development Center (NPRDG) , S^an Diego, in 
their eff^tts\to develop effective procedures fof* improvlng^the validity of 
Individual personnel selection decisions -based on accqrate measures of job- 
relevant performance. NPJRDC has an ongoing program to develop and explait ^ 
N^vy enlisted performance evaluation formats which will b^e effective in hold- 
ing down the pile-^^i^^pf marks at xthe high end of the marking scale, and itf ^ 
achieving' a d'istrllptlon of m^rKs that tapers off sufficiently at. thfe high end 
of the scale iix order to permit greater differentiation; |:hu^ making, evalu- 
ations more useful, especially when'siriall selection opportunities are in- 
volved. ^'^'^ An accurate and timely measurlji of each individtial*s on-job per- 
formance is essential if ^alid d0cisions are to be ffiade in selectipg .personnel 
•for advancement, duty assignment, training, or qttality retention. Suchy 
measure is one of the best^ indications of how wel^ th.e individual ^l^jll^ 
in other or. future assignments. However, effective use of performance mea- 
sures Ts severely limited due to the lack of perfbrmaace data in formats re- , 
spon^ive to the needs of the decision makers* The probleiri is' par;:tculai^y 
acute when these decision makers ar4 members of selection board? who must 
review in a short span of time narrative evaluat:^on data for thousahda of 
candidates. The seriousness of .this problem can be attests to by the faci:\ 
that some 14,000 candidates for promotion must be reviewed atinually by the E8- 
E9 selection board.. This number represents the top 75 percent of all candi- 
dates, the lower 25 percent having^ been eliminated by a screening procedure. . 
The problej^ is even more seYlous at the lower My grades; - An E7 selection ^£ 
board was '^tablished in 1973 whose taSTk is to review annually the records .of ^ 
some 20,000 enlisted candidates fdr promotion to chief petty officer. This 
number of candidates represents the top SOopercent of the eligible population, 
the bottom 50 percent having been eliminated by a screening f^rpcedure. 
. - ■ . . ■ ^ ' ■ . ^' 

The workload facing these ' selection **toards* is massive and to dat-e narra- 
tive comitients on -the performance evaluation forms have not been exploited in 
anyJlsystematic manner because narrative text resists easy analysis. Yet there 
seems tp be a great deal of .jlif f erentiating Information in^these narrative^ 
comments* that could substantially ai?d selection boards in bhoosing the most 
qualified candidates for promotion. This, then, was the task tfeit R-K Researfeh 
andy System Design took on ityan initial pilot study. ^ 

The Pilot Study ' '» / 

In the pilot study of the narrative sections of 224 Navy performance 
evaluations for senior, enlisted personnel .In Pay Grade E7,,U was determined 
by content analytic techniques that It Is possible to dlf fetentd,ate between, 
the performance of typical and superlative chief petty officers based on the 
■'substantive content of Evaluation Reports."* The results of this pilot study 
strongly suggested that there ar^e stable differences among the performance 
characteristics of chief petty officers In the various portions of the upper „ 
half of the marking scale on Performance of Duty that ati reflected in narra- 
tive statements written by evaluators. Prior to embarking on the Initial ^ 
pilot study, If was assumed that differences In marks' between the upper and 



lower 'halves of the marking scale were readily reflected in' narrative state- 
ments. * Howler, in order to address the* study to the realities and difficul- 
ties facing selection boards, who jmat make their selectipns only from 'a 
tmiformly high quality group of candidates, ^NPRDC provided R-K Research and^ 
System Design with a truncsrted data set comprising indif^iduals marked only in * 
the upper half of thfe' marking scale. The sample jjiata^hen were divided into 
thrjee cariteriotir groups--^--Uppet, Middle, and. Lowerji^gp^orresponding to -three 
portions of the' upper half ^pf the marking scal«i on Perf oinnance ,of Duty (the 
cr^Lterion vatiable). This truftcated data set required a much more rigorous . 
'anal^y^ical approach than would have been require* fol: a nontruncated data set. 
# • » ■ • 
In the pilot jyavestigation, NPRDC selected a sample of 224 EyaluationRe- . 
ports for senior enlisted personnel in Pay Grade E7# including 144 Aviation ^ 
Electronics technicians (AT's) and 80 Boiler Technicians (fiT's). All 224 
Evaluation Repoirts were dr^wn from the' ^op half of the marlcing ^fi^le on i9A- ^ 
PERFORMANCE OF DUTY of Evaluation Report F<i;ga^ NAVPERS 1616/8 (see Figure 1). 
Thip form subsequently has beten/replaccd by anothe'r form that can be scanned 
by an optical character reader; however, the content bf the^two forms is 
essentialiy the aam6. the ptlot study sample of 224. Evaluation Reports .was. . 
selected to contain three discontinuous criterion groups corresponding to t^e 
Up^er,rMldd3Le, and Lower portions of t^e upper half of ^:he markittig scale on 
igA^ERFORMANCE OF DUTY. ^ ^ . *. • . ' ' 

"# ■ ' • ' * • ■ " 

An indexing vocabulary**c6ns±^ting of 29 descriptive labels was deviaed to 
encompass the substantive conteAt of the narrative sections of Evaluation, Re-" 
poi;ts. These 29 ±n4ex terms fell into three major areas MANAGEMENr FUNC- 
TIONS, SKILLS AND ABILITIES, and PRODUCTIVITY AND ACHIEVEMENT. .Utider eacli'of 
thes^ headings there "were more detailed term^ such as PLANNING, TECHNlCAlft, ▼ 
SKILLS, arid AWARDS AND PUNISHMEl^T,' providing the indexer with a 3-leyel hier- 
*archy of descriptive *lab els from which to choose^ Each sentence of narrative 
text in the pilot gtudysample wds read carefully and„ ^here appropriate, 
divided into segments corresponding to the assignment of specific index terms. 
However, it is not ent>i/gh tq^rflmply label a narrative statement with the mosrf: 
appropriate index* term since the statement may have been a highly positive, ' 
quite, positive, neutral, quite' negative, or highly negative one. Therefore, a 
weigtv£ing scale containing^ six degrees xjf f avorableness/mnfavorableness 'was 
devised based on the range of adjectives and adverbs that occur in narifative . . 
tex|: of this kind. The indexing procedure that was used was the following: 
The narrative text of each Evaluation Report was read, segmented into dlHtinct 
statements, And each statement wa9 assigned one or tiiore indej^^ terms from the • 
set of 29 possible choices. Each ternf selected also was assigned a numerical \> 
weight from +3 to -2 depending upon the nature of the adjecti\fe8 or adverbs • 
used as modifiers in the statement.^ When the entire narrative text ''of the - 
Evaluation Report had been indexed, tlje indexing decisions that had }yeen made 
were recorded on a sj>ecial indexing form. v^A set- of 67 quantitative variables 
s^ibsequently was derived from the indexing form used to>record 'the content 
analysii decisions. Profiles or vec1:ors of these 67 valfies then were prepare 
for all of the Evaluation Reports contained in the pilot, study sample. Sepa- 
rate profiles vere complied for the Evaluation Section (19R)" and the Justifi- 
cation Section (19S) of e«||ft Evaluation Report. The statistical analyses that 
were, performed on the quantitative data extracted from the pilot study content 
analysis supported the hypothesis that narrative performance evaluations do 
contain lnformatioj[i useful to personnel selection boards in -discriminating 
between typical att<f superlative chief petty officers. j 
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Evaluation Report Form NAVPERS 1616/8 (a 75 percent 
photo redtiction of the original form). 



•'8:+, 



The Second study . ^ ' ^ , ^ 

The findings from the pilot study were considered to be prpvocatlve 
enough to warrant further Ix^yestlgatlon, Therefore^ a second itudy was^em-^ 
barked upon to attempt to ctojfil^s validate the ptllot study results on 222. new 
Evaluatlon tReports for senior enlisted men In the same two occupational spe^ 
clalties (138 AT' s and 8A BT's) tha,|; '^ere rep^resented in the pilot study 
sample and to extend the content analysis to 222. Evaluation Reports for senior 
enlisted men In two dliEferent .occupational specialties^ (60 Commlss^ttymen 
[CS's]. and 162 Radiomen [RH's]) than those lnve8tlgste<l in the pilot j^tudy in 
order to test the genexallzablllty of the content analytic tecJinlques devel-- 
oped earlier.^ As a further reflnetnent, the cross validation and gencrallza-* 
tlon samples of Evaluation Reports were analyzed Without any knowledge of th^. 
individual's Relative poiition in the upper half of the marking scale on 
Performance of Duty (the criterion variable). In the pilot study the -crite- 
rion data were made available early in the study, th^is introducing the pos&i- 
bllity that this knowledge subconsciously might have Influenced the content 
analysis' that was performed. This, factor was controlled for in the second 
study by withholding the ctiterlon information until the content analysis of 
the narrative texphsd beeki completed. 

»■ ' * • . . ' > ■■ ^ - 

In the secdnd study a series of more sophisticated an4 comprehensive 
statistical analyses was performed on the quantitative^ data^i^Xtracted from the 
cottterf£^ analysis, resulting in -.the following important* findings* It W^s pos- 
sible/ to index the cross validation sample in the blind J|||fithout knowledge of 
crlt^erlon group membership, and achlev^ as 'gopd; classif lotion accuracy as was 
achieved with the pilot study sample where criterion group membership was 
known to the indexer. Further, it was shown that better ciasslficatidfi into 
the three criterion groups was achieved when the two occupat:^<^nal specialties 
represented in the pilot study sampla and the cross validatioii sample were . 
treated separately. These findings suggest that classif'ication procedures 
based on the content analysis methodology developed' in thfs research should be 
tailored to 'specific occupations. In addition, it was shown that the content 
analysis methodology developed initially on the pilot study sampla- consisting 
of was generalizable to a new sample consisting of two different 

occupational specialties^ viz* , OS's and RM's. 

Also of concern in the pilot study were the issu'ee of ireliability and 
tralnablllty, albhbugh the scope of the mall Initial reseag^ti effort diH not 
permit these aspects tO be studied Iji any iubstantlftjL way.'^^^Therefoife, in de- 
signing the -second investigation thesa issues were; dealtt with by including a 
reliability study whoie objectives were twofold J (1) to determine the level 
of agreement b^ween palts of Individuals both of whom Independently would 
perform a content analysis of the'^ame corpus of Evaluation Reports, and (2) 
to investigate If^ ndhresearchers^ could ^be trained successfully to apply the 
complex, lengthy indexing procedure developed in the pilot s^dy.^ Kappa, 
weighted kisppa, and product'^-monient correlation were the three statistics used 
to measure agreement between the experienced indexer and two xellability 
ind^xers Itt thei^ assignm,ant of index terms and ^eights to the narrative text 
of a small corpus of Evaluation Reports. The value of the vai^ious agreement 
statistics that were computed ranged from .62 to .89 in this first reliability 
study. ' The initial expectation was that it would be extremaly difficult to 
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train nonrese'arch-orlented individuals to consistently Index the narrative 
sections of Evaluation Report forms using the complex content' analysis method- 
ology that had been developed In the pilot- study. The surprising result was 
l;hat In only six training sessions a quite respectabj.e level of agreement was 
achieved. This Is a significant f lading ^becaus^t suggests, that Navy and - • 
civilian operational personnel also can be trained to consistently apply 
•content analytic techniques. , * C 

• \ The Third Study 

In ^ follow-on investigation^ to the pilo't study and the second study; 
two tasks were- performed. The first task was' to try to develop valid, short- 
cut methods of indexing the narrative content of Evaluation Reports that would 
ex^tact the differentiating information contained in evaluative comments in a 
slm^e but reliable fashion, hopefully achieving as good or nearly as good 
classification accuracy as the longer, more complex -indexing procedure devel- 
oped Initially. ' Two short-cut methods of indexing the narrative content of 
Evaluation Reports were developed,, one a rational condensation, of the entire 
4?riglnal hierarchy of 29 index terms into a new set of 15 compressed terms, 
and the other a 15-term subset of the original hierarchy of 29 terms chosen on 
the basis of their early selection by the ste|>wise discriminant analysis 
prc^ess. The two short-cut Indexing methods, .although not achieving the 
classification accuracy >6'f, the priginal lengthy indexing procedure which had 
more variables available for the stepwise discriminant analysis process, did, 
however, achieve an* acceptable level of classification perfofmancein compari- 
son to the longe^, more complex indexing methodologys Of the twoshort-cut 
methods, the ratlon&l condensation indexing method was preferred -since it 
tracked the lengthy method more faithfully /in the selection bt discriminating 
variab'-les. Further, the rational condensation method examines all of the 
information contained in^a narrative performance evaluatjton whereas the statis- 
tically selected subset method Ignores certain portions df th$ narrative text. 

The key variables in differentiating between the performance of superla- 
tive chief petty officers ^nd their slightly^iess qualified colleagues were 
the adjS:tives and adverbs that an evaluator uses to describe the performance 
of the individual H^at it being evaluated; the range of skills and abilities 
that an individual marcLfests; and the following specific demonstrated capabi- 
lltlcB: Management and iupervisory ability; ekill in lead itig and directing 
his men; ability to organize his work Area and to staff it properly; ability 
to plan hle/workload and take any corrective measuree necessary to compensate 
for unforeseen obstacles to good performance; the ability to^resent an effec- 
tive image of hie work force to *pther components of tl^e Navy^nd to the civil- 
ian community; Bktll in communiqating effectively with others; a cooperative, 
and responsive way of performing his- Job duties; a creative, resourceful, and 
Innovative ^approach to his work; the drive and atamlna to perfofm well under 
tiring or adverae circumitances; hlft level of Intellectual functioning; pro- 
fessional ^and technical competence in his '.occupational specialty; his level of 
productlv;Lt^ and ^achievement; and recognition of hie assets and potential by 
hla subordinates, peers ^ and superior officers • 
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In the second task performed as part of the follow-on ixiyestigation to 
the pilot study and the second study, the original inter-indexer reliability 
study was extended^ in order to ej^cidate aore fully the issue of reliability 
o£ the complex, lengthy indexing procedure. In the extension of the reliabi- 
iity study, the various agreement statistics ranged from .56 to .83, similar 
in magnitude to t^e results obtained in tlfe first reliability study. Once 
again, %he heartening finding was that In #nly si'x .training s,ession8 a quite 
respectable level of agreement among indexe^s* was achieved. . ^ 

' , ' . The Fourth Study 

1^ 6bvlous'next step in this ffesearch was to cross validate the superior 

short-cut indexing technique the rational condensation method on other 

occupational specialties and on other pay grades than thosja that; had been stud- 
ied earlier (viz., AT's, BT's, CS's, and RM's in Pay Grade E7). During calen- 
dar year 1974, the narrative sections of a sample of 300 Evaluation Reports 
for enlistea personnel in Pay Grades E5 and E6 wete indexed using the rational 
condensation short-cut method. This data base consisted of performance evalu- 
ations on enlisted personnel in six occupational specialties, only one of 
which (R£tdioman) overlapped the four occupational specialties already studied: 
" A- _ ' - 







N 


AD 


-T Aviation Machinist)^ Mate. 


45 


DC 


- Damage Controlman 


30 




- Electronics Techni<^ian . 


^9 


PN 


- Personnelman 


66 


RM 


- Radioman. 


51 


SK 


- Storekeepet 


39 






TOTAL . 300 



Stepwise discriminant analysis Was used to determine how well the quantitative 
variables derived from the short-cut content analysis^ of the har^ative text 
could classify each individual evaluated into correct criterion, group. Each 
of the six occupational specialties represented in. the 'sample and each of the 
two pay grades were analyzed separately. This investlgatiojci was intended to 
show if Jthe rational condensation short-cut indexing procedure .is generaliza- 
ble to Pay Grades E5 and E6 and, to occupational specialties other than those 
studied thus far. *The results of this investigation are* presented in Section 
2. 

Using a subsample of the E5-E6 data base, a third reliability study was 
conducted in order to be certain, that consisteixcy among several indexers can 
• be' taught and achieved In their iijterpretation and application of the rational 
condensatioai. short-cut indexing method. The level' of agreement^ between^ each 
Pof the four reliability indexers and the experi-enced indexfer who trailed them 
was determined by the same statistical procedures used in the two earlier 
/reliability studies in order that comparisons couj-d be made among the three 



reliability st^udies of the magnitude 4f agreement that wafi achieved. This in- 
vestigation was intended tieT' lay the foundation for . a training curriculum that 
may be u^ed in the "futur^ to train Na^ and civilian -operational personnel in 
the application of^^-the^ntent analysis methodology.. The resdlts of the third 
reliability ^^44idy are presented in Section 3, * ^ 

The results' of the flrat twp reliaj^ility studies suggested* the possibi- 
lity that it may be as important to consider the issue of Jftternal consistency 
for a single indexer as to measure the^level of agreement that i can be achieved 
among several indexers. It seems reasonable to assume that although there may 
be slight diffei^ences between two indexers in how they apply a particular in- 
dexing procedure, a more important consideration is thaf they cptisistently use 
their own individualized interpretation of the indexing rules and conventions. 
One then might expect that regardless of which individualized interpretation 
was used to' index a particular data base, a similar • leveij. of t:lassif ication 
agreement with the criterion of on-Job' performance could be acliieved. This is 
an important area to study because the findings mdy point to the necassity.to 
use only one indexer for a particular data base if optimum extraction^ of 
differentiating information is to be obtained. ^ ^ 

In order to shed some light on this issue ^ a second indexer independently 
reindexed the cross validation and generalization samples using the origiTial 
lengthy indexing procedure. Thus, an exact replication of the indexing p6r- 
formed^by the first indexer in her content *ana lysis of the cross Validation 
and generalization sampl.e^ was carried qut independently. The accuracy of ' 
classification into correct criterion group achieved by. each of these two in- 
dexers was compared in order tS determiicie if both indexers wprking separately 
with their own individualized interpretation of th^. indexing rules and conven- 
tions could achieve comparable classification results . In addition, thfe ^ 
possibility that two indexers sharing the indexing of the same data base can 
achieve as good classification results as either indexer indexltig the entire 
data base alone w^ explored. The findings from these comparisons are dis- 
cussed^ in Section 4. " 

A careful 'comparison of the indexing, doding, keypunching, and subsequent 
computer processing time required to apply tfhe rational condensation short-cut 
indexing procedure and the original lengthy indexing procedure to a small sub- 
sample of the E5-E6 data base was made. This comparison provides data needed 
for assessing the economic feasibility of adding information extftacted from , 
narrative comments into a composite score for predicting an enlisted man s 
potential for assuming the managerial responsibilities of the next higher pay 
grade. The results of this ana^-ysis are described in Section 5. 

In Section 6 the areas of investigation to be undertaken In the final 
year of this research project are identified. 



SECtlON '2. CROSS VALIDATION OF THE RATIONAL CONDENSATION SHORT-CUT 
INDEXING PROCEDURE ON THE E5-E6 FLE^T TRIAL DATA V 

* ■ ' .. 

Introduction 

• " , ■ ^ - . . - • ■ 

In a previous' study undertaken as part of this research proiect, two ap-' 
.proaches to streamlining the original lengthy indexing procedur^ were de- 

• vised. 6 In the first approach the original hierarchy of 29 Index terms was 
compressed into a rational 'condensation iconsisting of 15 terms.- The rationale 
for this condensation grew out of extensive indexing experience ' and is based 
on management theory. The compression was achieved by combining those terms 
in the original hierarchy that logically bfeiong together in management pracr 
tice? or that tended to be conf^sed with each other in the actual indexing -of. 
the pilot study, cross validation, and generalization data bases. This "ap- 
proach, cabled 'the rational condensation, includes all of the information 
contained in- the original set of 29 index terms,, but extracts this information 
in a more' efficient, less confusing, and simpler f&shion. . 

• The Second approach to streamlining the compiex indexing metjipdology , 
•called statistically selected s#set, capitalized on the findings restiltitig 
froni the, various stepwise discriitilnant analyses that 'Uere performed originally • 
on the pilot study, cross validation, and generalizatipn samples. Plots of 
the classification accuracy achieved over the history of the .discriminant 
analysis procedure revealed that the most useful information in discriminating 
between superior chief pett/ officers and tlieir slightly less qualified col- 
leagues is contained In the variables G&lectea,, initially.' Therefore, a subset 
of approximately one-third of the initial set 'of 67 qtfantitative variables 
derived from the original' indexing form was determined,* based on the order in 
which these variables were selected into the discriminant functions for the 
four occupational specialties represented in the pilot study, cro^s valid a- 
tion. and generalization data bases, .i.e., AT's, BT's, CS's, and RM' s. 

In this earlier. study, 6 these two short-oat indexing procedures were? used 
to index the- narrative performance ©(valuations of 668 Navy enlisted men in Pay 
*Grade E7. The two methods proved t6 be comparable in classifying each indivi- 
dual Into correct; criterion group for the four occupational specialties. 
Neither method demonstrated an advantage over the other In ease of , implementa- 
tion. Therefore, since the rational condensation short-cut indexing method 
examined' ail of the information contained in a narrative . performance evalua- 
tion in contrast to the statistically selected subset method which takes Into 
consideration onlj^ portions of the narirative text, thus subjecting it to 
indexing oversights, the rational condensation method was chosen as the supe-. 
rior short-cut indexing ptocedure. ^ _ 

* The obvious next step in this research was to cross validate the short- 
cut indexing technique that proved to be superior in classifying the three E7 
data bases on other occupational ratings 'and on other pay grades. A set ot 
usable fleet trial* data were available at the Navy Personnel Research and 
Development Center (NPRDC) that had been generated in the process of testing a 
number of experimental forms for measuring ^bn- job performance for Pay Grades 
E5 and E6.3' Figure 2 shows an example of one of these forms. The experi- 
mental forms' differ in the, number 6f scale points uSed and the type^ of scale 
descriptors employed. However, their Evaluation and Justification Sections 
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FIRST and SECOND CLASS PETTY/OFFICER EVALUATION REPORT 

N"PTRl-.SD EX 5/6-.)o. ]R 



I. DESCRIPTION OR DUTIES ANP ACTIVITIES DVRtHG THIS R6PORTIN(fPE1?IOD 

CiNCLUDE DEPLOYMENTS AND EtJUCATIONAL ACHIEVEMENTS) 




■I' 



WORKSHEET 




T 



INITIATING OFFICIAL 



REVIEWING OFFICIAL 



REVIEWING OFFICIAl: 



REPORTING OFFICIAL 



EVALUATION SECTION 



C0>4PARE RATEE WITH ALL OTHERS OF HIS RATE 
KNOWN TO YOU. . " 



• ANY MARK IN TOP BOTTOM TWO COLUMNS 
V REQUIRES INQtyJDUAL JUSTIFICATION IN 
BLOCK 2(t ■ 



2. INDIVIDUAJL PRODUCTIV.ITY 



3. DIRECTING-^ 

4. CQtfjySfLING 



5, COOPERATION 



6.TLEXIBIUTY 



7- RELIABILITY 



8. OVERALL EVALUATION 



i/mTUAut.\ NE>I«LY 



E F F ECT I V E 



UNSATISFACTORY 



EXTREMELY EFFECTIVE 
ANP EXCELS OVERT 



ATMen& or same hate 



4, 



a , 

tu 

o 



U6HTeaMij^gr 



9. PERSONAL APPEARANCE 
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9 z 



22 MAM^(L'AST, FIRSTrMID^^^ OR l|(|ITIAL) 



SEM> ANNUAL □transfer QoTHER FROM 1^ L^^ 



Figure 2i Example of an Experimental First anti Second Class 
Petty Of fleer Evaluation Report Form (a 75 percent 
27 P'^^^^ reduction of the original form). ^ 



^S, rVALUATION COMMENTS: (Irulude v«rbol e«pre^tion oo<l ^mv»c»s to shipmotes, Oes^rihc whot yatee ji^f compliihed (or foiled 
to occompl^^h) or who^ ie$uhed from rote*'* perlormonc**. • ' 




. CLARJFJCATION OF TERMS 

Rot«e: Th« p«rsort b«(ng •voluoted. Rote: Roting ond poy grod*. ^ 
InnoyotivS Contributions:(Bllc 15): improvement tn procedures or 
mecbonisms creoted or developed by rotee.< 

evaluaS^ion items 

2. Individuol Productivity: Ratee's demonstrated technical compe- 
tence ond own worit output, includingf individuol contribution to 
group effort, but not including the work of persons rotee 
supervises. 

3^ Directing: Influencing othel*.! to accomplish a.iob^ 

4. Counseling: Assisting and encouroging subordinates in self- 
development ond to • fovorobie 4jsposition' toword the Naval 
Service. _ 

5. Cooperbtion: Promoting hormonious woricing relotionships ond 
teom effort^ 

6« Flexibility. Accomplishing work under changes in personnel, 

fob content, objecti/ves, or woricing conditions. 
7; Reliability: Corrying out tasks without direct supervision 
• or monitoring. 

8. Overoll Evaluation: General volue to the Novol Service. 

9. Personal Appeoronce; Dress ond grooming, on ond off duty, 
which reflects on the Naval Service. 

tff. Conduct: Be^oviori on ond off duty, whicTi reflects on the 
Novol Service. 



, EXCERPTS FROM SECTION 3410150, SUPERS MANUAL 

^''It Is. desired that the member's divisibn officer or appropriate chief 
^ petty officer moke the initial evaluation. The evoluatian shall hm 
based on the specific period of time invp|vl»d «hd reviewed ^'for^ * 
opprovol through the cnoin of commohd.'^ (poro. (3)) 

'*'*The completed Form shall bi si^Ttei^ by the commanding officer, 
except thot the commonding^officer may euthorize the exect/'tlve 
officer or deportment heod to sign provided such officers ore of 
the grade of LCDR above, or equlvelent grad« officer of onother 
service." (poro. (9}(b)) 

"Evaluations mugt be based objectively on the member's demon — 
strot4id perfo^monce and his abilities as ^compared to established 

Navy itendords and the performance of hfs contemporerles 

It is necessary that a member's shortcomings, sucn os alcoholism 
or other unreliability prdduclng deficiencies, be reported. Such 
deficiencies con be orvitol importance in the selection of members t 
for duty assignment, advancement, ftc.*^' (pqro. (4)) 

"Where memorondum entries of o meritorious or derogatory noture 
hove been mede in th» service record e.g., on pages 6, 9 or 13, 
the eveluotion should be corrected by on emount considered appro- 
priate in those traits which pertain to the entry." (pero. (10)) 

Block 25 sholl contain a definite statement as to the mi^mber's 
abilities in self— expression, orollv and in writing, ond commond 
of the English longuoge. (poro. (i)) 



26, JUSTIFICATION COMMENTS; (Use only to document any morks in the TOP/BOTTOM two columns of the Evoluofton Section, 
blocks 2 thru 10. Avoid the use of objectives which would simply be parroting the evaluation marks.) 



Figure JExample of an Expetimen^al First and Second Class* / 
(ContO ^etty Officer Evaluation Report. Form (a 75 percent ^ 2 
photo reduction of the original f orm) ^ 



are virtu^ly the same as these two sections in NAVPERS 1616/8 (see Flgute/l)-.* 
This extensive^ data base of fleet trial tlat a, which Included useful cr iter Ibn 
data as well, consisted of evaluations on enlisted personnel -in scjtven Occupa- 
tional specialties, only one of which (Radiotnan) : overlapped . the four rat>Lngs 
already studied. The fleet trial data base afforde4 an immediate pppprtimlty 
to cross validate the rational condeu&ation short-cut indexing procedure, ' 

• The/rirst E5-E6 Fleet Trial Sample 

A method for selecting a representative sample of these ^even occupa- 
tionar specialties from the E5 and E6 fleet trial data base, divided equally 
among three criterion groups based on on- job performanjce' (viz, , Upp 
Lower), was devised in collaboration with NPKDC, Mj^^ sample 
of Evaluation Reports representing all seven ocq^ia5fc^|||^^ specialties was 
selected using this sampling methodology* This it^piii sa^ 
as the first E5-E6„fleet trial sample or FleW Trl^$; Sba^le 1 in order tro dis- 
tinguish it from a second similar sample currently being Btfudied (see Section 
6)1 Only nine Hospital Corpsman cases were drawn ittt:o this first sample be- 
cause of their sparse representation in the overall fleet trial data base. 
Therefore, although these nine Evaluation Reports were itudexed using the 
rational condensation short-cut method, they were set aside for latet atialysis 
as part of the second B5-E6 fleet trial s^ple and replaced by nine Damage 
Controlman cases in order to augment this rather small occupational group. 
Table 1 portrays the ^composition of the first 300-case E5-E6 fleet trial 
samp la used in the investigation being reported here. , - 



TABLi 1 . 



COMPOSITION OF THE FIRST 300-CASE E5-E6 FLEET TRIAL SAMPLE 



Occupational 
Specialty ^ 


N In 
E5 • 


N In 
E6 


Sum 


■ >». 

AD - 


0 

Aviation Machinists Mate 


,18 


27 


45 


DC - 


Damage Controlman 


15 


15 


30 


ET - 


Electronics Technician 


• 48 


21 


"69 


PN - 


Personneltnan 


27 


- 39 


66 


RM - 


RadiQiaan 


18 


33 


51 


SK - 


Storekeeper 


15 


• * 24 


39 


■1 


141 • . 


159 


300 



Derivation of the Quantitafive Variablei^ • 
Used in the Step^ise\Discrifninant Analps^s 

A set of 2^ quantitative variables wa#tferived f rdia the rational conden- 
satip'n short-cut indexing form used in the Content janali^sls of the E5-E6 fle^t . 
trial narrative performance evaluations' (see Table 2)- The; first 15 variables ' 
represent the weighted frequency of e4cb ind[eX term used to index a particular, 
section of narrative text» Variable Iftjis^t^^ the 15 weighted frequfenr?;;- 

' eies. Variables 17 through 21 repre^lnt the^ frequency Counts over' the. entiif^:^ ^ 
rational condensation indexing form for the j^sslgnment of weights applied .tO S . 
modifying adjectives ^nd adverbs used by axi, evaluator in a partlctilar section 
^ of narrative text. Variable 22 is the tot ^1 plumber of index ternfe of the 15 
available that were used to Index a partictkl^^f section of narrative text. 
Variable 23 is the total nximber of words in >the section oK, narrative text that 
wa^ indexed. The reader is referred to Ap^epdlx A and to an- earlier technical 
report^^ for a more detailed description of the^rational condensation short^Ctit 
indexing procedure and the derivation and qb^titif^^cation of Variables used in 
performing the stepwise discriminant analysis^ - / . 

^ro^iles or vectors of these 23 variables were constructed for all of the 
Evaluation Reports contained itt the first 300-case fleet trial sample based on 
the indexing decisions resulting from the content analysis o| the narrative 
comments.' Separate profiles were compiled for the Evaluation Section and the 
Justification Section of eadh Evaluation Report. All profiles were entered 
onto IBM coding forms and keypunched. The indexer'did not know to which 
criterion group an Evaluation Report belonged While she was making her index- 
ing decisions • This information was added to the IBM coding forms just before 
,they were keypunched, v 

The first fleet trial sample then wsis analyzed by Program BMD07M in the 
library of Biomedical Computer Programs^ , at thevJJCLA Health Sciences Computing 
Facility, This program performs a multiple discriminant analysis in a step- 
wise manner. At each step one variable is' entered into the set of discrimi- 
nating variables or a variable is deleted if its F value becomes' too low. At 
the option of the user, a classification matrix is computed and printed after ^ 
those steps specified by the user.*^ " * 



^ Results 

Figures 3 through "14 depict the accuracy of classifying J:he six occupa- 
* tional specialties represented in Fleet Trial Sample 1 into correct criterion 
grou? based on on-job performance as reflected In* the narrative comments 
^ written in the Evaluation Section and the Justification Section. In all of 
these figures it *can be seen that at the initial ^Step a substantial portion of 
the classification problem is solved. Each curve portrayed in these '12 f ig-^ 
ures continues to climb slowly bu^ asymptotically after the initial step; ^ 
however', perfect classification performance is* never reached. 

The best cfassiflcation that wa^ achieved on the Evaluation Section 
narrative comments and the Justification Section narrative comments for the., 
sijft occupational specialties represen^d In Fleet Trial Sample 1 is shown in 
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TABLE .2 



DEFINITION OF THE 23 QUANTITATIVE vWaBLES -DERIVED 
FROM THE RATIONAL CpNDENSATION INDEXING FORM 



Number of 
Variable 

1 
2 



A 
5 

6 . 
7 
8 
9 

10 
11 • 

12 

13 . • 

14 ' 

15 
16 
•17 
18 

19 
20 
21 
22 
23 



Deafcrlptlon of Variable 

Weighted Frequency of Menjiilon of ilANAG^MENT'iftll'^IONS 

Weighted Frequency of MerJt^on^ LEADERSHIP AND 
' ■ ^ DIRECTING- 

Weighted Frequency of f,Merielon of ORGANIZATION AND [ 

• ■ STAFFING '■ i ' 

Weighted Frequency of Mention of PLANNING-CONTROLLlNG ^ 

Weighted Frequency, of Mefitlon of REPRESENTATION 

Weighted Frequency of Merition of SKILLS AND ABILITIES 

Weighted Frequency of .Mention of COMMUNICATION 

Weighted. Frequency of Mention of CONDUCT AND ATTITlfoE 

Weighted Frequency of Mention of COOPERATION AND 

V.,' RESPONSIVENESS 

Weighted Frequency of Mention of ENDURANCE AND. MOTIVATION 

Weighted Frequency of Mention of CREATIVIT*^:A|;n) 

INITIATIVE " - 

Weighted Frequency of Iteiition of INTELLECTUiii 

Weighted Frequency of liention of PROFESSIONAL AND 

^ TECHNICAL ^ILLS 

Weighted Frequency o£ 'Mention of PRODUCTIVITV AND 

ACHIEVEMENT 

Weighted Frequency of Merttlon of RECOGNITIOIJ 

^ ' .»),.■ 
Sum of Variable!? 1 through 15 • 

Total Number of +3 Weights - 

Total Number of +^Weighti8 

Total Number of +1 Weights 

Total Number^ of -1 Weights 

Total Number o£ -2 Weights ^ * j 

Total Number of Index Terms ^Used 

Total Number of Words in Narrative Text V' 
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Figure 3. Accuracy of Classifying the Fleet Trial Sample 1 AD's (N»45) 
into Correct Criterion Group (Ejyaluatio'n Section - 19R) . 
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Figure 4. Accuracy of Classifying the Fleet Trial Sample 1 AD's (N«45) 
into Correct Criterion Group (Justification Section - 19S) . 
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.Figure 5. Accuracy of Classifying th'fe Fleet Trial Sample, lo DC * 8 (N-SO) 
into' Correct Criterion Group (Evaluatiqn Section - 19R) • 
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Accuracy of Classifying the Fleet Trial Sample k DCT^s (N-SO) 
Into Correct Criterion Group (Justification Section - 19S) . 
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'.Figure 7. Accuracy of Classifying the Fleet Trial Sample 1 ET's (N-69) 
, ' ^ Into Correct Criterion Gtoup (Evaluation Section - 19R) . » 
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Accuracy of Classifying the Fleet Trial ^Sample 1 ET's 
into Correct Crlterlox)i Group (JustlS^lcatlon Section - 
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19S)'. 



^2? 
8 « 
It 

»si to 



70 
65 
60 
SS 

tSO 
«»5 
MQ 
.35 
30 

25 
20 
15 
10 



Fleet Trial Sample 1 PR's 




I 2 3- ^ S 6 7 8 9 X0 11]^2131ifl5l617l819 2021 
Step Number in the Stepwise Discriminant Atialyslm 



, Figure 9. ^ Accuracy of Cli^ssifying thie^ Fleet Trl^^^ 

\ into Correct Criterion Group (Evaluation Section - ISB)V. 
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Figure 10. Accuracy of fclasslfylng the FleefTrlai Sample 1 FN'a (N-66) 
Into Correct Criterion Group (jtiptiflcatloh- Section - 19S). 
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Figure 11, Accuracy of Classifying the Fleet Trial Sample 1 RM's (N»51) 
ilnto Gorre^t Criterion Group (Evaluation Section - 19R). 
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Figure 12, Accuracy of Classifying the Fleet Trial Satpple 1 RM*8 (N=5l)^ , 
Into Correoj: Criterion Group* (Justification Section - l^S). 
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Figure 13. ^ Accuracy of Classifying the Fleet Trial Sam]ple 1 SK*a (N-39) 
into -Correct Criterion Group (^Evaluation Section - 19R) , 
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Figure, 14. Accuracy of Classifying the Fleet Trial Sample 1 SK's (N-39) 
Into Correct Criterion Group (Justification Section - 19S) . 
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Tables 3 through 8. The underlined diagonal elements of the classification 
matrices portrayed in these six tables correspond to the agreement between the 
statistical classification itkto criterion group and. actual criterion group 
membership; the of f-diagonal elements represent disagreement. The total 
number of statistical classifications matching actual classifications' is 
ob-tained by summing the diagonal elements of each -matrix, shown in .tj|iese six • 
tables as the underlined diagonal sum. The step in the discrfminant analysis 
at which this best classification was achieved for the Evaluation^ectlon and 
the Justification Section also is shown in these tables .and corresponds to the 
maximum point in the curves Shown for each occupational specialty in Figures 3 
through 14. It should be pointed out that 'this presentation of the discrimi- 
nant analysis results assumes that the criterion of 'actual gro^p membership is 
perfect where in fact the possibility does exist that some 'of the members of 
the sample were given inflated marks on on-job t>erf ormance , and consequently, 
were assigned to an incorrect critejpion group. ' • . 

A recapitulation of the clas<ftfication accuracy achieved for the six 
occupational specialties represented in the first fleet ttial sample using the 
rational condensation short-cut indexing procedure is p^vided in Table 9. 
Class if idation accuracy ranged from 67 percent to 90 petcent. In earlier 
content analysis studies 5* 6 at the E7 pay grade level /it was found that 
better classification performance was achi^eved in the stepwise discriminant ^ 
analysis of the Justification Section narrative conments compared to the 
Evaluation Section narrative comments. -This findd/g was not corroborated at 

the E5-E6 pay grade level?. In only two E5-E6 octupational specialties PN 

and SK was the classification performance hepter for the Justification 

Section. For the E5-E6 DCs, ET's, atid RM's better classification perforaancet 
was achieved in th6 stepwise discriminant analysis of the Evaluation Section, 
and for the E5-E6 AD's the classification accuracy was the same for both 
sections of narra.tive comments. 

The classification accuracy achieved in the stepwise discriminant analy- 
sis of the first 300-case E5-E6 fleet trial sample is less than that achieved 
in earlier content analysis studies of the narrative performance evaluations 
for four occupational specialties in Pay Grade E7.5'6 Several reasons may 
help explain these results. First, in the lower pay grades the job duties 
performed by enlisted personnel are less demanding, more routine, and not so 
Lnagerial in nature as the job duties performed by chief petty officers in 
Wy Grade E7. Consequently, there may be less critical observational data 
available upon which to base a performance evalujftion, reflecting itself in 
narrative cotnnents that may lack th°e necessary substance to differentiate, 
between good arid superlative first and second class petty officers. The 
indexer's subjective Impression was that the narrative text of performance 
evaluations typically is shorter for the lower pay grades, and Table 10 sub- 
stantiates this impression for the Justification Section. The total number of 
words in the narrative 'text -of the Evaluation Section on the average Is quite 
similar for the thr^e pay g'rades on the Upper-end Middle criterion groups. 
Only on the Lower criteirion group is there a noticeab^ gradient for the- 
Evaluation Section from fewer to more words on the avStage arf pay grade 
increases i and even then the average number of words for the pilot study E7 s 
Is almost identical to that of the fleet trial E6's. . However, on the Justi- 
fication Section the gradient^ is. very clear-cut for all three criterion 
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, TABLE 3 « 

BE§T CLASSIFICATION INTO THE THREE CRITERION GROUPS 
FOR THE FLEET TRIAL. SiSllPLE 1 AD 'a (N-45) USIN'G THE 
RATIONi^ CONDENSATION SHORT-CUT INDEXING PROCEDURE 
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TABLE 4 

BEST CLASSIFICATION INTO THE THREE CRITERION GROUPS 
FOR THE FLEET TRIAL SAMPLE 1 DC'fl (N-30) USING THE 
RATIONAL COIJDENSATION SHORT-CUT INDEXING PROCEDURE' 



EVALUATION SECHtON - 19R 

Step 17 Classification by 
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■ TABLE, 5 

BEST CLASSIFICATION INKL THE THREE CRITERION GROUPS 
FOR THE FJ^EET TRIAL SAMPLE 1 ET*8 (N=69) USING THE , 
RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 
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Step 14 Classification by 

Discriminant Analysis 
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- TABLE 6 • 

BEST CLASSIFICATION INTO THE THREE CRITERIOO^JPS 
FOR THE FLEET TRIAL SAMPLE 1 PN's CN-s66) USING THE 
RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



EVALUATION SECTION ^ 19R 
Step 19 



o 

•H J3 

U (0 

2 g UPPER 

^ I midIdle 

g^g LOWER 
a M 



Classification by 
Discriminant Analysis 

UPPER MIDDLE LOWER 

18 4 0 

3 116 3 

V 1 6 15 

Diagonal Sum « 49 

(49 * 66 X 100 ^« 74%) 



JUSTIFICATION SECTION - 19S 
Step 15 
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TABLE 7 



BESTSejiASSlFICATIOK INTO THE THREE ORITERION GROUPS 
FOR THE FLEET TRIAL SAMPLE. 1. RM's (N=51) USING HJB 
RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDUJIE 
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TABLE 8 • 

BEST CLASS lElCATION INTO THE THREE CRiiEERION GROUPS , 
FOR THE FLEET TRIAL SAMPLE 1 SK's (N«39> USlNGf THE 
RATIONAL CONDENSATION SHORT-CUT INDEXING iPROCEDURE 
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TABLE 9 



RECAPITULATION. OF THE CLASSIFICATION ACCURACY ACHIEVED 
FOR THE SIX OCCUPATIONAL SPECIALTIES REPRESENTED 
IN THE FIRST FLEET TRIAL SAMPLE USING 
THE RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



- Sample/Occupational Specialty 
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Size 
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Accuracy 
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graupaV with the narrative text of justification comments fpr the lower two 
^ay grades being considerably shor4:er on the average. Jf few^jt worda.also 
indicate less substantive content , then there , is less potentially differen--. 
tiating information in the narrative commentia written by eyaluatdrs for en- . 
listed personnel in Pay Crades E5 and E6> * / . 

Another possible exjplanation to help account for the poorer elassifica-^ 
tion accuracy achieved in the stepwise discriminant analysis of the first ES- 
E6 fleet trial sample compared to that achieved in earlier content analysis 
studies at the E7 pay grade level is that the sample sizes in the six Occu- 
pational specialties represented in the first' E5-E6 fleet trial aamp^e are 
quite ^mall in comparison to the sample fizes studied earlier for thfe f our E7 
occupational specialties. Enlarging the sample sizes for the £irstfE5~E6 . 
fleet trial sample occupational specialties might result in* classification 
performance of higher accuracy* A third possibility is that the six occupa-^ 
tional sf>e<;ialties represented in the first E5-^E6 fleet .trial sample consti- 
tute job categories that afford less differentiating performance dajta than the 
four occupational specialties represented in the E7 samples studied .earlier, 

. ■ ' •• ' ■ ■ , ^ * ^ ■ ' ; ■■ ■ 

^ Ther« was only one occupational specialty in .qomejon in the E5-E6 and E7 
samples— Radioman (EM). The class if icat;ion results on the Evaluation Section ^ 
were comparable in the two sain^les. For the Evaluation Section; 42 of the 51 
E5-E6 M'*s were classified correcltly (82%) compared 1:o 131 of the 3,62 E7 BM'a 
(81%). "However, for the Justification Section, 41 of the 51 E5-^E6" BM's were 
classified correctly (80%) compared to 144 of the 162 E7 EM*s' (89%), -The 
better classification accuracy on the Justification Section achi^ed for the 
E7 RM's compared to the E5-E6 RM*s most likely is a reflection of the lengthier 
narrative text in justification comments written about enlisted personnel at 
the higher pay grade level. 

In earlier research,^ it was shown that better classification is achieved- 
if each occupational specialty is analyzed sfepiSr^tely rather than if all of 
the occupational specialties represented in a sample are c6inbined for the 
stepwise discriminant analysis. This earlier finding was corroborated, in the 
analysis of the first E5-E6 fleet trial sample, , The six occupational special- 
ties represented in the sample were combined to form two subaamples, oner 
consisting of l4l E5's and the other consisting of 159 E6^fi^. Figures 15 and 
16 depict the accuracy of classifying the Fleet Trial Sample 1 E5's into 
correct criterion group. for the Evaluation Section^and for the Justification 
Section. Again, a substantial portion of the classification problem is solved 
at the fi%st step. In Figure 16, however, the final clasEk^tcation perfor- 
mance is essentially the same as that demonstrated at the initial step. The * 
best classification accuracy that Was achieved on the Evaluation Section and 
the Justification Section of the Fleet Trial Sample 1 E5*S is shown in Table 
11. Of the I4i E5's represented in the first E5-E6 fleet trial sample, 91 
(65%) were classified correctly on the Evaluation' Section^ this maximum classi- 
fication accuracy occurring a^t Step 8. On Ihe JuStificatiojdi Section, at Step 
la a maximum of 95 (67%) of the 141 ES's were classified correctly. In most 
instances, this classification performance Tfe inferior to that achieved when 
the six occupational specialties were analyzed separately • 
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1^3 If 5 6 7 e 9 10 11 12 13 1^X5 16 17 1.Q192021 ' 
Step Number In the Stepwise Dlsc^trimlnant Analydld 



tlguve 15. Accuracy of Classifying the Fleet Trial Sample T^ES's (N*141) 
into Correct Criterion Group (Evaluation Section >* 19R). 
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Fleet Trial Sample 1 E5's 
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Accuracy of Classifying thf Fleet Trial Sample 1 E5's (N-141) 
into Correct Crijterloh Group (Justification Section - 19S>. 
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TABLE 11 . * 

BEST CLASSIFICATION 'INTO THE THREE CRITERION GROUPS 
FOR THE FLEET TRIAL SAMPLE 1 E5's (N-141) USING THE 
RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



EVALUATION SECTION - 19R 

Step 8 Classif icatdLon by 

Discriminant Analysis 



a ou 
o 

U CO 

0) U 

4J (U 



4J O 
O M 





UPPER 


MIDDLE 


l'ower 


UPPER 


28 


16 

K 


3- 


MIDDLE 


. 12 


31 


A 


LOWER 


7' 


8 


32 



Diagonal Sum = 91^ 
(91 V 141 X 100 = 65%) 

■ ( ■ 



JUSTIFICATION SECTION - 19S 

•V 

Step 10 Classification by f 

Discriminant Analysis 

' • ° 1 
UPPER 



0 o. 

O -H 
•H J= 

S £ UPPER 
^ g MIDDLE 
§ LOWER 

4J O 

u u 



MIDDLE LOWER 
38 '6 '3 
3 10 3A 

»0 . 0 47 

^ Diagonal Sum = 95 
(95 * 141 X 100 «= 67%) 



ERIC 



U^PPER 



C Pu 

0 -H 

U CO 

d) M 

1 S MIDDLE 



TABLE 12 

BEST CLASSIFICATION INTO THE THREE CRITERION GROUPS 
FOR THE FLEET TRIAL SAMPLE 1 E6's (N-159) USING THE 
RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



JUSTIFICATION SECTION - 

Step 20 Classification by 

Discritninant Analysis 



0 q« 
o -H 
•H xi 
u co- 
0) u 
U 0) 

u| 

(d Pu 
4J O 

q M 
<5 a 



EVALUATION SECTION - I9H 

Step 5 Classification by 

Discriminant Analysis 



UPPER MIDDLE/ LOWER 
'36 12 5 

. 12 32 9 

15 29 
Diagonal Sum = 97 





UPPER 


MIDDLE 


LOWER 


UPPER 


43 


6 


A 


MIDDLE 


A 


29 


20 


LOWER 


0 


A 





(97 V 159 X 100 =%1%) 



Diagonal Sum = 121 
(121 * 159 X 100 = 76%) 
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/ 
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Figutfs 17 andAs depict the^lE^uracy of classifying the Fleet Trial 
Sample a E6^d into correct criterion group for the Evdluation Section and for 
the Justification Section. A substantial portion of the classification prob- 
lem is solved at Step 1 in both of these figures; however', the classification 
performance shown in Figure 18 for the Justification Section is sufferior to 
that shown in Figure 17 for the Evaluation Section. The best classification 
accuracy that was achie^fed on the Evaluation Section and the Justification 
Sec€ion of the Fleet Trial Sample 1 E6's is shown in Table. 12 • Of the 159 * 
E6?'s represented in the first ES-E6 fleet trial sample, 97 (^%) were classi- 
fied correctly on the Evaluation Section,' this maximum classification accuracy 
occurrlnfe at Step 5. On the Justification Section, at Step 20 a m^ximjmi of 
121 (76%) of the 159 E6's' were classified correctly* This classification per- 
formance for the E6 subsample is somewhat better than that achieved for the E5 
Bubsample but still less t^axl that achieved for most of the six occupational 
specialties analyzed separately 

It is of particular Interest to note that most of the mlsclass^i^catlons 
made by the stepwise discriminant, analysl/s procedure, as judged by, the crlte" 
rion of on-- job performance^ were In the direction of classlf^ng an individual 
into a lower crl.terion group than the one to which he actually belonged* • If 
it is acknowledged that the criterion of on- job performance may be imperfect, 
then -what the stepwise discriminant analysis procedure appears to 2>e doing is 
to sift out the individuals who might have been assigned to a higher criterion 
group because of inflated marks 'on the writer ion of on- job performance. If 
this speculation proves to be true, then the stepwise discritdnant analysit" 
procedure results in flagging for consideration only those individuals who 
manifest the highest job performance potential based on the narrative comments 
written by their evaluators. Application of this statistlaaf^echnlque then 
would help to narrow the field of candidates for advancement, duty assignment, 
training, or que^lity retention to only those individuals potentially the most 
qualified. 

L 

An interesting set of results is revealed by an examination or which 
variables were selected by the Stepwise discriminant analysis program for the 
first 15 steps in each of the computer runs that were made. Tables 13 through 
18 show the results for the six occupational specialties represented in the 
first E5-E6 fleet trial sample. The results for the Evaluation Section and 
the Justification Section are presented separately in each of these six %^le3. 
A careful perusal of the findings presented in these tables reveals that the 
variables selected early tend to be different between the Evaluat^n Section 
and the Justification Section and among the six occupational specialties. 
These differences may be attributable to real variation among ^ job groups with 
regard to the skills and abilities required to perform one's Job duties well, 
or they may only reflect aberrations attributable to the smallness of some of 
the sample sizes. These findings point up the need to cross validate the 
results of studies based on small samples in order to determine which discrim- )k 
inating variables are constant over more than one sample. If the same set of 
variables emerge as the most discriminating ones when two or more samples are 
cotnbin'ed to produce larger sample sizes^ then more confidence can be placed in 
the results. An /extension of the investigation being reported here is dis- 
cussed in Section 6. - ' / 
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Figure 17. Accuracy of . Classifying the Fleet Trial Sample l^E 
into Correct Criterion Group (Evaluation Section - 
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Figure 18. Accuracy of Classifying tha. Fleeti Trial Sample 1 E6's (N==«159) 
into Correct Criterion Group (Juf^tif ic^tion Section - 19S) . 
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TABJ^ 13 

JkRIABLES SELECjaiD BY THE STEPWISE DISCRIMINANT ANALYSIS PROGRAM ' 
T AT STEPS 1 THROUGH 15 FOR THE FLEET TRIAL SAMPLE 1 AD's 
VsiNG THE RATIONAL CONDENSATION SHORT-CUT INDEXING • PROCEDURE 



Ev&luatlon Section 



Justifi0btlon Section 



vf of PRODUCTIVITY tAND " 
ACHIEVEMENT 

Total Number of -1 Weights 

wf of PROFESSIONAL AND 
TECHNICAL SKILLS 

wf of COOPERATION AND 
RESPONSIVENESS 

wf of REPRESENTATION 

wfv-of LEADERSHIP AND 
DIRECTING 

wf of CREATIVITY AND INITIATIVE 

wf of PLANNINGr CONTROLLING 

wf of ENDURANCE AND MOTIVATION 
Total Number of 2 Weights ^ 
wf of CONDUCT AND ATTITUDE 

wf of IljTELLECTUAL FUNCTIONING 
Wf of SKILLS AND ABILITIES 
Total Number of Index Terms Ue^d 
Total Number of 1 Weights 



Total Number of Index Terms 
Used 

Wf of MANAGEMENT FUNCTIONS 
wf of RECOGNITION 

wf of ENDURANCE AND MOTIVATION 

Wf of CREATIVITY AND INITIATIVE 
Total Number of 3 Weights 

wf of PLANNING-CONTROLLING 

wf of COOPERATION AND 
RESPONSIVENESS 

wf of REPRESENTATION 

wf of* INTELLECTUAL FUNCTIONING 

\^ of PRODUCTIVITY AND 
ACHIEVEMENT 

wf of CONDUCT AND ATTITUDE 

l^al Number of 1 Weights 

wf of SKILLS And abilities 

Total Number of Words in Text 
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TABLE 14 



VARIABLES SELECTED BY THE STEPWISE DISCRIMINANT ANALYSIS PROGRAM 
AT STEPS 1 THROUGH 15*" FOR THE FLEET TRIAL SAMPLE 1 DCs 
USING TEE RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



Step 

X 

1 

' 2 

3 
4 
5 



. 7 
8 
9. 
10 
11 

12 
13 
14 
15 



Evaluation Section \ . 

Total Number of -1 Weight^s 

wf of INTELLECTUAL FUNCTIONING 

wf^of PLANNING-CONTROLLINQ 

wf of COOPERATION AND , 
RESPONSIVENESS 

'wf of ORGANIZATION AND STAFjM 
HTG 

wf of PRODUCTIVITY AND 
ACHIEVEMENT 

wf of CONDUCT AND ATT^ITUDE 

wf of MANAGEMENT FUNCTIONS 

wf of ENDURANCE AND MOTIVATION 

wf of RECOGNITION 

Total Number of Index Terns 
Used • 



w£ of REPRESENTATION 

Number of 1 Weights 
Number of Words in Text 



Total 
* » 

Total 



Justlfiaa tion Section 



Total Number of Index Terms 
Used 

wf of COOPERATION AND 

RESPONSIVENESS * 

wf of LEADERSHIP AND 
DIRECTING 

wf of PLANNING-CONTROLLING 



wf of ORGANIZATION AND STAFF* 
ING 

Total Number of 1 Weights 



wf of RECOGNITION 

wf of ENDURANCE AND MOTIVATION 

Total Number of Words in text 

wf of CREATIVITY AND INITIATIVE 

wf of COMMUNICATION 
(laiSi:' step) 



wfof READERSHIP AND 
DIRECTING 



ERIC 



r50 



« TABLE 15 ' . • 

VARIABLES SELECTED BY THE STEPWISE DISCRIMItlANT ANALYSIS PROGRAM 
AT STEPS 1 THROUGH 15 FOR THE FLEET TRIAL SAMPLE 1 ET's 
USING THE RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



Step 



Evaluation Section 



Justification Section 



1 Total Number of -1 Weights 

2 w£ of ENDURANCE AUD MOTIVATION 

3 Sum of Variables 1 through 15 

A wf of COOPERATION AND 

^ RESPONSIVENESS 

5 V wf^-of pREATIVITY AND INITIATIVE 

6 wf of COMMUNICATION 

7 wf of PROFESSIONAL AND 

TECHNICAL SKILLS 

C 

8 wf of SKILLS AND ABILITIES 

9 wf of CONDUCT AND ATTITUDE 

10 wf of INTELLECTUAL FUNCTIONING 

11 ■ Total Number of -2 Weights 

12 Total Number of 2 Weights 
13' . wf of REPRESENTATipN 

14 Total Number <|f Words in Text\ 

15 . STotal Number of Index Terms 

Used 



Total Number of Words in Text 

wf of PRODUCTIVITY AND 
ACHIEVEMENT 

''a 

wf of RECOGNITION 

Total Number of -1 Weights 

wf of MANAGEMENT FUNCTIONS 

Total Number of 1 Weights 

wf of INTELLECTUAL FUNCTIONING 

wf of COOPERATION AND 
RESPONSIVENESS 

Sum of Variables 1 through 15 

Total Number of 2 Weights 

Total Number of Index Terms 
' Used 

wf of CREATIVITY AND INITIATIVE 

wf of ENDURANCE AND MOTIVATION 

wf of CONDUCT AND ATTITUDE 

wf of ORGANIZATION AND STAFF- 
ING 



ERIC 



51 
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TABLE 16 

VARIABLES • SELECTED BY THE STEPWISE DISCRIMINADT ANALYSIS PROGRAM 

AT STEPS 1 THROUGH 15 FOR THE FLEET. TRIAL SAMPLE 1 PN's • 
. USING THE RATHttJAL CONDENSATION SHORT-CUT INDEXING PROCEDURE- 



Bvalnation Section ' Justification Section 



1 Sum of Variables 1 through 15 Sum of Variables 1 through 15 



2 


Total 


Number of -1 Wjaights 


Total Number of Index Terms 
Used 


3 


Total 


Number of Index Terms 
Used ^ 


To^al Number of 2 Weights 


4 


wf of 


CONDUCT AND ATTITUDE 


Total Number of -2 Weights 


5 


wf of 


ENDURANCE AND MOTIVATION 


Total Number of -1 Weights 


6 


wf of 


REP,RESENTATION 


wf 


of ORGANIZATION AND STAFF- 
ING 


7 


wf of 


INTEI^ECTUAL FUNCTIONING- 


. wf 


of MANAGEMENT FUNCTIONS 


9 


Totar 


"Number of Words in Text 


wf 


of INTELLECTUAL FUNCTIONING 


9 


wf of 


RECOGNITION 


wf 


of CREATIVITY AND INITIATIVE 


10 


wf of. 


SKILLS AND ABIlTlTIES 


wf 


of SKILLS AND ABILITIES 


11 


wf of 


PRODUCTIVITY AND . 
ACHIEVEMENT 


/ wf 


■«» 

of COOPERATION AND 
RESPONSIVENESS ' 


12 


wf of 


PLANNING-CONTROLLING 


wf 


of COMMUNICATION 


13 


Total 


e 

Nutnber of 2 Weights 


wf 


of PROFESSIONAL AND 
TECHNICAL SKILLS 


14 


wf of 


ORGANIZATION AND STAFF- 
ING 


.wf 


of LEADERSHIP AND ' 
DIRECTING 


15 


Total 


Number of 1 Weights 


• wf 


of RECOGNITION » 
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TABLE 17 

VARIABLES SELECTED BY THE STEPWISE DISCRIMINANT ANALYSIS PROGRAM' 
AT STEPS 1 THROUGH 15 FOR THE FLEET TRIAL SAMPLE 1 RM's 
USING THE RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDURE 



Step 



Evaluation Section 



Justification Section 



3 
A 
5 



8 
9 
10 

11 

12 
13 

14 

15 



wf of PRODUCTIVITY AND 

ACHIEVEMENT ■ 

Sum jDlt^ Variables 1 through 15 

wf of CONDUCT AND ATTITUDE 

Total Number of Words In Text 

Total Number of Index Terms 
Used 

wf of SKILLS AND ABILITIES 
wf of PLANNING-CONTROLLING 

wf of ^COGNITION 

wf of MANAGEMENT FUNCTIONS 

wf of REPRESENTATION 



wf of ENDURANCE AND MOTIVATION 

Total Number of 3 Weights 

wf of ORGANIZATION AND STAFF- 
ING 



) 



tf of LEADERSHIP AND 
D'IRECTING 



Total Number of Index Terms > 
Used 

wf of REPRESENTATION 

wf . of CONDUCT AND ATTITUDE 

wf of ENDURANCE AND MOTIVATION 

wf of PRODUCTIVITY AND 
ACHIEVEMENT 

yff of MANAGEMENT FUNCTIONS 

wf of ORGANIZATION AND STAFF- 
ING 

Total Number of -1 Weights 
wf of 
wf of 



INTELLECTUAL FUNCTIONING 

COOPERATION AND 
RESPONSIVENESS 



wf of PROFESSIONAL AND 
TECHNICAL SKILLS 

Total Number of Words In Text 

.wf of SKILLS AND, ABILITIES 

wf of RECOGNITION 



wf of COOPERATION AND 
RESPONSIVENESS 



wf of COMMUNICATION 
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TABLE 18 

variables! SELECTED BY THE STEPWISE DISCRIMtlJANT ANALYSIS EROGBAM 
AT STEPS 1 THROUGH 15 FOR THE FLEET TRIAL SAMPLE 1 SR's 
USING THE RATIONAL CONDENSATION SHORT*CUT. INDEXING PROCEDURE 



Evaluation Section 



{Justification Section 



Wf of COMMUNICATION 



wf of LEADERSHIP AND 
DIRECTING . 

wf of PRODUCTIVITY 'AND 
ACHIEVEMENT 

wf of MANAGEMENT FUNCTIONS 



wf of INTELLECTUAL FUNCTIONING 

J ■ 

wf 0-f RECOGNITION • ' 

Total Number of 3 Weights 

wf of COOPERATION AND 

RESPONSIVENESS « 

Total Number of 2 Weights 

wf of ENDURANCE AND MOTIVATION 

'wf of ORGANIZATION AND STAFF- 
ING 

Total Number of Words in Text 

wf of PLANHING-CONTROLLING 

wf of PROFfiSSIONALwAND 
TECHNICAL SKILLS 

Total Number of Index Terms 
Used 



Total Number of Index Terms 
Used . 

wf of COMMUNICATION 



wf of SKILLS AND ABILITIES 



' wf of PROFESSIONAL AND 
TECHNICAL SKILLS 

wf of COOPERATION AND 
RESPONSIVENESS 

wf of LEADERSHIP AND 
DIRECTING 

O 

Total Number of Words in Text 
Sum of Variables 1 -through 15 



wf of RECOGNITION 

wf of PRODUCTIVITY AND* . 
ACHIEVEMENT 

f 

wf of CREATIVm AND INITIATIVE 



Total Nutober of 1 Weights 



wf of MANAGEMENT FUNCTIONS 
wf of INTELLECTUAL FUNCTIONING 

.wf of PLANNXN6-CONTR0LLING 




'-■■iff-::' 



In tables- 19 and- 20 thd variables selected by the stepwise discriminant 
analysis program at Steps 1 through 15 kte shown for the first fleet trlg^l 
sample E5's aijd E6's,^ Againj^ the results for the Eval^atib and the 

Justification Section are |>resented separately^ tSWen all 

specialties ere considered together* at e^h of t^ it 



7 



is of Interest to note that the same variable was selected firM for the^^^^ 
and the E6's, albeit a different- variable f^or the two sections of narrative /! 
comments. For'the Evaluation Section^ Total Number of -*1 Weights wa$ the 
first variable selected fcPr both E5's and E6*s« flSr the Justification Sec- 
/ tion, the first variable delected was Total Number of Inde» Term^ Used, The 
. first variable selected is extremely in^ortan^ since at least half pf the 
. classification problem tends to he solved at the first step for these two pay 
. grades regardless of which section of narrative edmments ia bMng analyzed. 
Particularly noteworthy is the selection by the stepwise diacriiflinafift analysis 
program of the variable, --Total Number of index Terms ^Used^ for the Justifica-^ 
tion Sfettion. In earlier content analysis studles^^^ at the Ji7 psy grade 
level,, without Exception itl^e first variable selected for the^ Justification 
^Section was Total Number of Index Terms tfsed. This variably reflects tlje 
'-variety of specific areas of an individual's performance that the evaluator 
chose to comment on, ai^ is measured by the number of dlffereht index terms . 
chosen by the liid€Xei>^to encompass the narrative content; This finding ^indi- 
cates that the range of skills and abilities' that a petty officer manifests is 
a key factor in his superior performance as narrated by the evaluator In the 
Justification Section* 

_ In order to gain a better feeling for which variables .may prove to be the 
most discriminating ones in further* studies of these six ES-E6 . Occupational 
specialties based on larger sample sizes, Tables 21 and 22 were prepared. 
These two tables present a siimmary of the variables selected by the stepwise 
.discriminant analysi{.s program for all six occupational spectialtles arranged in 
order by frequency of choice and early selection. Table 21 depicts the re«- 
sults for the Evaluation Section; Table .22- shows the results for the lustifi- ' 
cation Section*. The way that these two tables were constructed was first to 
note Tihose variables that within the first 15 steps were selected for all six 
occupational specialties, then those selected for five of the six occupational 
V specialties^ and so on down to being selected within the f irst 15 steps' for ^ 
okly one of the sjx occupational specialties. , Then, within each> level of 
frequency of choice the variables were ranked by how early they were selected, 
based on the sum of the steps for all six occupational specialties* For 
example, in Table 21 it can be seen that only two variables wer^ selected 

within the first 15 steps for all six occupational specialties ^weighted fre-- 

quency (wf) of ENDURANCE AND MOTIVATION and Total Number bf Index Terms Used.:; 
Since wf of ENTDURj^CE^ AND MOTIVATION tended to be selected earlier than Total 



Number of index Terms Used 
was assigned a rank of one 



(as reflected in^ total sum of 46 versus 63), it 



Although the results shown in Tables 21 and 22 are only tentative, they 
do allow soihe interesting preliminary observations to be made.. First, for 
both sections of narrative comments all 23 quantitative variables dprived from 
the rational condensation short-cut indexing procedure were selectejd ^within 
the first 15 steps by at least one occupational specialty, suggestlhg that all 



VARIABLES SELECTED BY • tHE STEPWISE DISCRXMINAIW ANALYSIS tROGB^M 
AT ST^S 1 THROUGH 15 FOR THE "FLEET jEMAL SAjI^LE 1 E5*s 
using' THE RATIONAL CONDENSATION SHORT-CUT INDEXING PROCEDORE 



, Evalmiiion Section 



Justifioatiaa Section 



yotal Number of *1 Weights 



wf of PRoDiicTiviTY em 

ACHIEVEMENT 

of PLANNING-CONTROLLING 

•H^f - ORGANIZATION ^MD STAFF- 
ING , 



wf of ENDURANCE AND MOTIVATION 



wf ■ of PROFESSIONAL AND- 
TECHNICAL SKILLS 

wf bf CREATIVITY AND INITIATIVE 



wf of RECOGNITION. ^ 
Total Number of 1 Weights 
wf of REPRESENTATION 
^ wf of COMlpJICATION 

w^ of INTELLE9TUAL FUNCTIONING 
wf of MANAfllBIST FUNCTIONS 
To taiy^ number of 2 Weights 



wf of LEADERSHIP AND 
DIRECTING 



Total Number o£ Index Terms 
Used 

wf of COMMUNICATION 



wf of REPRESENTATION , 
Total Number Of -1 Weights 



wf of PRODUCTIVITY AND 
ACHIEVEMENT 

Total Number of 2 Weights 



wf of LEADERSHIP AND 
DIRECTING 

wf of CREATIVITY AND INITIATI^ 

^- Total Number of 1 Weights 

Total Numbjer of Words in Text 

°' wf of MANAGEMENT FUNCTIONS . 

wf of B^COGNITION 

Total Number of -2 Weights 

. wf of ORGANIZATION MD STAFF- .. 
ING 

, wf of • ENDURANCE AND MOTIVATION 



TABLE 20 

VARIABLES SELECTED BY THE STEPWISE tilSCailMINANT ANALYSIS PROGRAM 
AT STEPS i THROUGH 15 FOR THE FLEET TRIAL SAMPLE 1 E&'s 
USING THE RATIONAL CONDENSATION SHOiff-CUi; INDEXING PROCEDURE 



Evaluation Section 



Justification Section 



Total Number of -1 Weights 
Sum of Variables 1 through 15 

wf of PLANNING-CONTROLLING 

wf^bf PRODUCTIVITY AND 
^ ACHIEVEMENT 

wf of jfouRANCE AlA MOTIVATION 

Total Number of 1 Weights 

Total Number of Words in Text 

Total Number of Index Terms 
Used 

wf of CREATIVITY AND INITIATIVE 



wf of LEADERSKiP AND 
DIRECTING^ 



Wf of REPRESENTATION 

wf of CONDUCT AND -ATTITUDE 

wf of COOPERATION AND 
RESPONSIVENESS 

Total Number of -2 Weights 



:wf of INTELLECTUAL FUNCTIONING 



Total Number of Index Terms 
Used 

wf of PRODUCTIVITY AND 
ACHIEVEMENT 

wf of CREATIVITYAND INITIATIVE 

wf of SKILLS AND ABILrriES ' 

Total'\umber of 1 Weights 

Total Number of -.1 Weights 

wf of INTELLECTUAL FUNCTIONING 

wf of PROFESSIONAL AND 
TECHNICAL SKILLS 

Total Number of Words in Text 

Sum of Variables 1 through 15 

wf of REPRESENTATION 
wf of COMMUNICATION 

wf of ORGANIZATION AND STAFF- 

.; ING 

wf of COOPERATION AND 

RESPONSIVENESS f 

Total Number of 2 Weights 



. -TABLE 21 

• SUMMARY OF- VARIABLES SELECTED BY THE STEPWISE DISCRIMINANT 
ANALYS.IS PROGRAM FOR THE SIX OCCUPATIONAL SPECIALTIES 
REPRESENTED IN THE FIRST E5-E6 FLEET TRIAL SAMPLE ^ 
ARRANGED IN ORDER BY FREQUENCY OF CHOICE AN|r EARLY SELECflON 
(EVALUATION SECTION - 19Rf 







Occupat 


lonal 


Special,tf" 






y Nam^ of Variable 


AD 


DC 


ET 


PN 


RM 


SK ' 


Total 


wf 'of ENDURANCE AND MOTI^I)^TION 


V 9 


•9 


2 


5 


11 


10 


4^ 


Total Number of Index Termk Used 


14 


11 * 


15 


3 


5 


15 


63 


wje' of PRODUCTIVITY AND 
ACHIEVEMENT 


1 

% . 


6 




11 


1 • 


3 


22 


wf of CONDUCT AND ATTITUDE 


11 


7 


9 




3 




34 


wf Qf COOPERATION AND 
RESPONSIVENESS 


4 


4 


4. 




15 


8 


35 


wf of INTELLECTUAL FUNCTIONING 




Z 




7 




5 


' 36 


wf of PLANNING- CONTROLLING 


o 
o 






X2 


7^ 




43 


wf of REPRESENTATION 


5 


12. 




6 


10 




46 


Total Number of Words in Text 




14 


14 - 


8 


4 


12 


52 


-Total Number of -1 Weights 


2 


1 


1 


2 






6 


wf of RECOGNITION 




10 




9 


8 


6 


33 


wf of LEADERSHIR AND DIRECTING 


6 


15 






14 


2 - 


37 


wf of SKILLS AND ABILITIES 


13 




8 


10 


6 




37 


wf of ORGANIZATION AND STAFFING 




5 




14 


13 


11 


43 


Total Number of 2 Weights 


10 




12 


13 




9 ' 


44 


Sum of Variables 1 through 15 






. 3 


1 


2 ^ 




6 


wf of MANAGEMENT FUNCTIONS 




8 


} 




9 


4- 


21 


wf of PROFESSIONAL AND 
TECHNICAL SKILLS 


3 




7 






14 


24 


Total Number of 1 Weights 


15 


13 




15 






43 


wf of COMMUNICATION 






6 






1 


7 


Wf of CREATIVITY AND INITIATIVE 


i 

- 7 




5 








12 


Total Number of 3 Weights 










12 


7 

s 


19 


Total Number of -2 Weights 






11 






11 



NOTE: The numbers in the table represent the step at which the 
variable was selected. The total is the sum of the steps 
for all six occupational specialties. 



58 



-3- 
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. ' TABLE 22 < 

^IMIARY OF VARIABLES SELECTED „BY THE STEPWISE DISCRIMINANT 
ANALYSIS. PROGRAM FOR THE SIX OCCUPATIONAL SPECIALTIES 
. REPRESENTED IN THE FIRST E5-E6 FLEEP TRIAL SAMPLE - 
ARRANGED IN ORDER' BY FREQUENCY OF CHOICE AND EARLY SELECTION 
(JUSTIFICATION SECTION - 19S) 



Occupational Specialty 





Name of Variable 


AD 


DC 


ET 


PN 


RM 


SK 


Total 




Total 


Number of Index Tem^? Used 


1 


1 


11 


2 


i 


1 


17 




wf of 


COOPERATION AND^ 
RESPONSIVENESS 


8 


2 


8 


11 


10 


5 


•44 




wf of 


RECOGNITION 


3 


7 


3 


15 


14 


9 


51 




wf of. 


MANAGEMENT FUNCTIONS 


2 




5 


7 


6 


: 13 


33 : 




Total 


Number of Words in Text • 


IS 

-< 


9 


1 




12 


7 


44 




wf of 


CREATIVITY AND INITIATIVE 


5 


10 


12 


9 




11 


47 


wf of 


INTELLECTUAL FUNCTIONING . 


10 




7 


8 


9 


. 14 


48 




wf of 


PRODUCTIVITY 'AND 
AClflEVEMENT 


11 




2 




5 

* 




28 




wf of 


ENDURANCE AND MOTIVATION*: 


k 


8 


13 




4 




29 


w 

»■> 


wf of 


ORGANIZATION AND StAFFlNG 




5 


15 


6 


7 




33 


Total 


Number of 1 Weights 


13 


6 


6 






12 


37 


V 


wf of 


COMMUNICATION 




11 




12 


15 


2 


40 




wf of 


SKILLS AJ^^ILITIES 


14 






10 


13 


3 


40 




tbtal 


Number br -1 Weights 






4 


5 


8 




17 




Sum of Variables 1 through 15 






9 


1 




8 


18 ' 




wf of LEADERSHIP AND DIRECTING 




3 




, 14 




6 


23 


Q 


wf of 


PLANNING-CONTROLLING 


7 


4 








15 






wf of 


PROFESSIONAL AND 
TECHNICAL SKILLS 








13 


11 


4 


28 




wf of 


CONDUCT AND ATTITUDE 


12 




14 




3 




29 




wf of 


REPRESENTATION 


9 








2 




11 




Total 


Number of 2 Weights 






10 


3 






13 




Total 


Number of -2 Weights 








4 






4 




Total 


Number of 3 Weights 


6 












6 





NOTE: The numbers in the table represei^t the step fit which the 
variable was selected. The total is the sum of the steps 
for all six occupational apecialties. 
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variables used in thW stepwise^ discriminant analysis have something to contri^* 
bute to solving the classification problem. Second, a particularly differen- 
tiating <?ariable is Total Number of Index Terms Used, attaining a rank of twO 
for tUpg Evaluation Section and a rank of one for the Justification Section. 
This variable indicates the range of skills and abilities that a petty ^pf f Icet^ 
manifests^ Third, the number and type of adjectives, and adverbs that an 
evaluator u^tes to describe the performance of the individual that/is being 
evaluated (as reflected in the various, weights) do not appear to contribute 
vety much to differentiating amonfe superlative first and second class petty 
officers and their slightly less qualified jffeers , whereas at the ET.pay grade 
level (ohief petty officers) these weights are important discriminators. / 
Fourth, there appears to be less critical observational data available at tfte-L_^ 
lower pay grades upon which to base a performance evaluation, as reflected in 
shorter narrative comments on the Justification Section. Pifth, when b6th 
sections of narrative comments are considered together, the more* discrlmi- . 
nating variables seem to be Total Number^ of Index Terms Used, weighted fre- 
quency of COOPERATION AND RESPONSIVENESS, weighted frequency of ENDURANCE AND 
MOTIVATION, weighted frequency of PRODUCTIVITY AND ACHIEVEMENT, weighted fre- 
quency of INTELLECTUAL FUNCTIOtJiNG, weighted frequency of RECOGNlflON^ and 
Total Number of Words in Text. At Pay Grades E5 and E6, it seems reasonable 
^ that a petty officer's cooperative and responsive spirit, his endurance and 
motivation, his productivity and achievement, his level of intellectual func- 
tioning, and the amount of recognition he receives for his on-job performance 
would be more significanr variables than leadership and management qualities 
that will be brought into play later in his career as he advances through the 
pay grade structure from junior to senior enlisted petty officer. A further 
investigation to c?Jtrdbo#ate or refute these i)reliminary conclusions is 
described in Section 6^ 
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:ection 3:; reliability study of the rational condensation short-cut 

INDEXING procedure ^ . 



Introduction 



In the pilgt^ contrent analytic study of the narrative sections of Navy 
performance evaluatioiis for se1ii#r enlisted personnel in Pay Grade E7, the 
issues of reliability and trainability were of concern, although the scope of 
the small initial research effort did nop permit these aspects to be studied 
in any substantial way. Therefore, in„designing the second Investigation 
these issues were dealt with by including an original reliability study whose 
objectives w'ere twofold: (1) to determine the level of agreement between 
pairs of individuals both of whom independently would perform a content analy- 
sis of the same corpus of Evaluation Reports^, and (2) Xo investigate if non- 
researchers could be trained successfully to apply the^omplex content analy- 
sis methodology developed in the pilot study. ^ Kappa, wiSighted kappa, and 
product-moment correlation were the three statistics used to measure agreement 
between the experienced indexer and two reliability Indexers A and B in their 
assignment of index terms and weights to the narrative text of a small corpus 
of 48 Evaluation Reports. The results of this original reliability study are 
presented in Table 23 in the upper left-hand quadrant labeled 1972 - Original 
Data Base: Lengthy Indexing Procedure. The value of the^varioiis agreement 
statistics ranged from .62 to .89 in this first reliability study. The ini- 
tial expectation was that it would be extremely difficult to train nonresearch 
oriented individuals to consistently index the narrative sections of Evalua-. 
t ion Report forms using the complex, lengthy indexing procedure that had been 
developed in the pilot study. The surprising result was that in only six 
training sessions a quite respectable level of agreement was achieved. This 
is a significant finding because it suggests* that Navy and civilian opera- 
tional personnel also can be trained to consistently apply content analytic 
techniques. L 



In the follow-on investigation to the pilot study atid the second study, 
the original inter-indexet reliability study was extended in order to eluci- 
date more fully the issue of reliability of the complex, lengthy indexing 
procedure. In the extension of the reliability study, the various agreement 
statistics ranged from .56 to\s^3, similar in magnitude to the agreement ob- 
tained in the first reliability study. These results also are shown in Table 
23 in the upper right-hand quad^nt labeled 1973 - Second Data Base: Lengthy 
Indexing Procedure and in the lower left-hand quadrant labeled 1973 " Original 
Data Base: Lengthy Indexing Procedure. In the second reliability study 
conducted in 1973, Reliability Indexers A and Bwere given a new and different 
set of 48 Evaluation Reports to index independent^ using the lengthy indexing 
procedure. This aspect of the second reliability study was Included in order 
to test the hypothesis that with additional training and indexing experience, 
the level of indexing agreement could be raised. Neither Reliability Indexer 
A nor Reliability Indexer B was able to irfcrease her level of agreement with 
the experienced indexer despite ,re fresher training in the complex, lengthy 
indexing procedure and the challenge to try to outdo her previous performance. 
HQwever, these two reliability indcxqrs felt that the data base indexed by 
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difficult to index than the Original reliability study data base, and this 
greater difficulty inherent in the narrative text may have masked any gain in 
indexing proficiency that might have been ^achieved by the additional trainings 
Another possible explanation >is that Reliability Indexers A and B may have 
already approached the upper boundary of their indexing skill, with additional 
training and experience contributing Very little to increasing their level of 
agreement with the experienced indexer. ^ 6' * 

c Additionally, in the second reliabilitj^ study conducted in 1973, two new 
reliability indexers were included, a male and a female, both in their sopho- 
more year in college. The two new indexers. Reliability Indexers X and Y, in- 
dependently indexed the same 48 Evaluation Reports that had been indexed in 
the first reliability study. These two individuals in essence were attempting 
to replicate the 1972 results, but they did not achieve as high a level of 
agreement with the experienced indexer as Reliability Indexers^A and B did in 
the initiaL 1972 study, probably because they were less motivated and not as 
deeply involved in the study as wer^ Reliability Indexers A and B who^lso 
were regular employees of R-K Research and System Design, performing a variety 
of clerical assignments in addition to their role as reliability indexers. 
However, the heartening finding in this extensioji olS.the original reliability 
study was that once again, in only six train^jjg sessions, a quite respectable 
level of agreement between indexers was achieved. 

Using a subsample of 48 Evaluation Reports drawn from the E5-E6 fleet 
trial data base, a third reliability study was conducted in order to cer- 
tain that consistency among several indexers can be taught and achieved in 
their interpretation and application of the rational condensation short-cut 
indexing method. The level of agreement between each of the four reliability 
indexers and the experienced indexer who trained them was determined by the 
same statistical procedures used in the two earlier reliability studies in 
order that comparisons could be made among the three reliability studies of 
the magnitude of agreement that was achieved. This investigation was intended 
to lay the foundation for a training curriculum that- may be used in the future 
to train Navy and civilian operational personnel in the application of the 
content analysis methodology- 

Methodology for the Reliability Study 
of the RatioAal Condensation Short-Cut Indexing Procedure 

A new training manual was prepared to explain and illustrate the proper 
utilization of the ratiotial condensation short-cut indexing procedure. This 
training manual i^reproduced in Its entirety in Appendix A. It contains a 
discussion of the management process as it applies to Navy enlisted personnel^ 
how the index terms used in the content analysis are to 'be quantified (weight- 
ed), special Indexing considerations, a summary of indexing rules to be follow- 
ed, 'and an alphabetical dictionary of the 15 index terms used in the rational 
condensation short-cut indexing procedure complete with definitions for each 
term and replete with examples of how each term in the dictionary should be 
used. 
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Four reliability indexers were trained by the experienced indexer in the 
application of the rational condensation short-cut indexing procedure using 
the (graining manual reproduced in Appendix A» As in the first ^o reliability 
studies, six sessions were employed to accomplish the training of the reliabi- 
lity indexers. These four indexers then independently indexed the narrative 
comments contained in a newly selected subset of 48 Evaluation Reports taken 
from the E5-E6 fleet trial data base. Oiice again, the Evaluation Section was 
separated from the Justification Section so that the narrative comments for 
each of these two sections of an Evaluation Report were not considered to- 
gether. This process resulted in a group of 96 randomizeB pieces of narrative 

text minidocvments — -that were indexed independently by each reliability 

indexer. Their indexing decisions then were compared to those of the experi- 
enced indexer who trained them and whose dejcis ion-making processes they were 
trying to emulat^. The same agreement Statistics that were used in the two 
previous reliability studies (kappa, weighted kappa, and product-moment corre- 
lation) were computed in order to determine if the^reliability of the rational 
condensation short-cut indexing procedure is comparable to that found in the 
two earlier reliability studies of the original lengthy indexing procedure. 
The teader is referred to earlier reports for a detailed exposition of how 
the three agreement statistics were applied in determining the reliability of 
indexing decisions arrived at by the various indexers participating in the 
first two reliability studies. 



Jesuits 

^' ... 

The kappa statistic^ was the measure of agreement used in analyzing the 
index terms assigned by the four reliability indexers and the experienced in- 
dexer in the third reliability study of the rational condensation short-cut 
indexing procedure. For each segment of narrative text, each indexer chose a 
term or terms from the list of 15 possibilities, or the decision Vas made that 
no term should be used. From a careful analysis of these indexing decisions 
for each reliability indexer compared to the experienced indexer, it was 
possible to compute the kappa statistic (k) for each of the four pairwise com- 
parisons. These results are shown in Table 23 in the lower right-hand quad- 
rant of the table labeled 197^ " Third Data Base: Rational Short-Cut Indexing 
Procedure under the column heading, Agreement on Tertns. 

Reliability Indexer A exceeded her performance ifH the ^973 reliability 
study and equalled her performance in the ^972 reliability study by achieving 
a kappa value of .89 in the selection of index terms. Reliability Indexer B 
exceeded h^r performance in both 1972 and 1973 by attaining a kappa value of 
.77. Reliability Indexer X also increased his performance over 1973 by 
achievH.ng'a kappa value of .77. From these results it is possible to conclude 
that consistency among several indexers can be taught and achieved in their 
interpretation and application of the index terms contained in the rational 
condensation short-cut indexing procedure as well as those contained in the 
lengthy indexing procedure. ^ 

Reliability Indexer Z was the only new indexer added for the 1974 study. 
The lower level of agreement of her selection of index terms with, those of the 
experienced indexer (k«=<i53) may be attributable to less indexing experience 
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than the other three reliability indeXets had, but more likely to the fact 
that personal problems interfered with her ability to do a careful job/ 
During the course of the 1974 reliability study, this indexer suffered a mis- 
carriage, was in an accident, and e^^perienced a death in hef >f atnily, events , 
which may account for the incomplete state of her work when she turned it in. 
She finally did find all of the missing materials and compl^eted all of her 
indexing judgments. However, it is quite clear that she ^ave less than a 
conscientious performance in thiJ3 study. Her agreement statistics are of 
interest nevertheless because th^ probably represent the kind or indexing 
performance that can be expected of an individual who* might not be particu- 
larly attentive or motivated* _ ^ , ^ 

Analysis of the level of agreement between the experienced indaXet and 
each of the four reliability indexers participating in the 197^ reliability 
study in assigning numerical weights to each index term selected, bashed on the 
modifying adjectives and adverbs, was performed differently than the analysis 
of the level of agreement in selecting the index terms themselve^ Selection 
of' the index terms in the three reliability studies constitute^ ^nominal 
scale whereas assignment of a numerical Slight to each index t^m selected wa|^ 
an indexing decision involving an ordinal scale. Therefore, more powerful 
agreement statistics could be employed. Since numerical weights on a scale 
from 3 to -2 were assignee} to each« index term selected, ^t was possible to 
compute a product-moment correlation coefficient (r^) between the weights 

assigned by the exp^ienced indexer and by each reliability indexer . In addi- 
tion, another agreement statistic, weighted kappa <k ),^^'^^ also was computed 

•* . 

to determine if it agreed with the results of ^ the correlational analysis.' The 
results of the correlational analysilg and the calculation of weighted kappa 
also are Shown in Table 23 in the lower right-hand q^adrant^of th? table 
labeled 197^ - Third Data Base: Rational Short-Cut Indexing Pf%cedure under 
the column .heading. Agreement on Weights. The results for weighted kappa are 
presented first, followed by^he correlation coefficient after the slash mark. 

Reliability Indexer A*s performance in assigning weights to index terms 
was approximately equal in the 1972 and 1973 reliability studies using the 
lengthy indexing procedure. However, in 1974 using the rational condensation 
short-cut indexing procedure she substantially increased her agreement .with 
the experienced indexer by achieving a weighted kappa of .88 and a correlation 
coefficient of .90. The same gain in agreement with the experienced indexer 
in assigning weights to index terms is evidenced in the results for Reliabi- 
lity Indexer B. Her resuVts in the 1972 and 1973 reliability studies using 
the lengthy indexing procedure were essentially the same, increasing to a 
weighted kappa of .77' and a correlation coefficient of .82 in the 1974 relia- 
bility study where the rational condensation short-cut indexing procedure was 
employed. Reliability Indexer X also demonstrated better agreement with the 
experienced indexer in assigning weights to index terras in the 1974 reliabi- 
lity study using the rational condensation short-cut indexing procedure com- 
pared to the 1973 reliability study which empldyed the lengthy indexing pro- 
cedure. This gain in indexing performance achievea^4?y Reliability Indexers A, 
B. and X possibly can be attjiributed partially to additional indexing experi-' 
ence. However, a more plausible explanation is that th)p new training manual 
prepared to instruct indexers in the proper utilization^f the rational con- 
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densation ^Mort-cut indexing procedure served its purpose well,^ The rational 
condensation short-cut indexing procedure is considerably easier to apply than 
the more complex, lengthy indexing procedure, and in addition the confusion 
over the selection of one or another index term in the lengthy indexing proce- 
dure has been eliminatjed in the rational condensation short-cut indexing 
method by combining those terms that tended to be confused or that logically 
belong together in management practice^ ^ The assignment of weights also has 
b^en facilitated by the new training manual in that an extensive list of ^ 
adjectives ami adverts falling at each point on the weighting 'scale is pro- 
vided (see Table A-2). In addition, tshe rules for assigning '^weights were sim-^ 
plified so that less decision making was required of the reliability indexers/ 
The rules for assigning weights were applied in a straightforward manner and 
there were fev?er areas of ambiguity. 

Reliability Indexer Z once again exhibited a lower level of agreement 
with the experienced indexer l,n assigning^yeights to index terms compared to 
the other three reliability indexers partlfcipating in the 1974 Reliability^ 
study, although her assignment of weights to index terms was better than her 
selection of index terms themselves. For all of the other reliability index- 
ers, judt the opposite situation resulted, that is, better agreement with the 
experienced indexer was exhibited in the selection df index terms rather than 
in the assignment of weights to the terms ^elected, ^However, it should be 
kept in mind in examining this finding that the three agreement stat^-stics 
reported in Table 23 are not directly comparable. Nevertheless, the values 
for wgight^ed kappa and the product-moment correlation coefficient are very 
simlla^ in all of the comparisons that were made. ^ 

In the computation of the various agreejnent statistics for* the 1972 and 

1973 reliability studies, those instances were not taken into account where 
both the experienced indexer and one of the reliability indexers had made a 
decision not to index a* particular segment of narrative text because the text 
comprised factual information rather than evaluative comments. Actually, 
these instances constitute agreement between the two indexers because they 
both made a conscious Judgment not to assign an ipdex term (or weight) as 
indicated by their placing the factual text in brackets. Therefore, in the 

1974 computation of the various agreement statistics, those instances where 
both the experienced indexer and a reliability indexer made a decision not to 
index a particular segment of narrative text were counted as agreements be- 
tween the two indexers. The agreement statistics that had been computed in"^ 
1972 and 1973 in the two earlier reliability studies were adjusted to include 
this type of agreement that had not been included previously. The resulting 
effect on the magnitude of t^e various agreement statistics wAs minimal, 
changing them in an upward direction by only -01, .02, or .03. This adjust- 
ment to the values of the 1972 and 1973 reliability study agreement statistics 
as shown in Table 23 accents for any minor differences between the reliabll-^^ 
ity study results presented in this technical report and In earlier reports. 
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In suiwnary, the conclusion that can be drawn from this third reliability 
study ,is that consistency among several Indexers in their interpretation and 
application of the rational condensation short-cut indexing procedure to the 
narrative coimnents contained in performance evaluations for naval enlisted 
personnel is high and better than that achieved with the lengthy indexing 
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procedure in two earlier reliability studies. In the 1972 and 1973 reliabil- 
ity studies of the lengthy indexing procedure, better agreement with the 
experienced indexer was exhibited in the selection of index terms compared to 
the assignment of weights to these terms based^ on the .modifying adjectives and 
adverbs used by an evaluator. In the 1974 reliability study of the rational 
condensation shqrt-cut indexing procedure, agreement with the experienced 
indexer was approximately the same "for the selection of ind^ terms^and the 
assignment of weights to these -terras (disregarding Reliability IndeAer Z) . In 
all three reUabillty stiidies, quite respectable levels of agreement>between 
the experienced indexer and the various reliability indexers were achieved in 
only six training sessions, indicating that Navy and civilian" operational 
personnel also should be able to learn to consistently apply the content 
analytic techniques developed Iri^ this rj^search project. To this end Appendix 
A of this technical report presents a training manual for the rational con- 
densation ffhort-qut indexing procedure, and. Appendix A of an earlier ^chnical. 
report 5 provides a training manual for the lengthy indexing procedure. 



\ 



67 



53 



ERIC 



SECTION 4. \ VALIDATION OF THE ORIGINAL LENGTHY INDEXING PROCEDURE 
MEAJJS OF A SECOND INDEXER - r 



Introduction 



The results of the ^irst two reliability studies suggested the possibi- ' 
lity that It may be as l%iportant to' consider the issue of internal conststency 
for a single indexeii as to -measure the level of agreement that can be achieved, 
among several indexers. It s6ems reasonable to assume tHat alttiough there may^ 
be slight differenpes b^^tweeh two ^.ndexers in how they apply a particular in- 
dexing procedure, d tnb?re important -consideration is thatthey consistently use 
their own individualized interpretation of the indexing rules and conventions. 
One then might expect/that 'regardless of which individualized interpretation 
was used to index a Particular data base> a similar level of classification 
agreement with the oYiterion of 6n-job performancij could be achieved. This is 
an important area tb study because the findings may point to the necessity to 
use only one indexer for a particular data base if optimum extraction of 
differentiating information is to be obtained. 

^ Methodology ^ 

In order to shed some light on this iss^e, a second indexer independently 
reindexed' the 222-cas^ross validation sample and the 222-case generalization 
sample using fhe-^ original lengthy indexing procedure. Thui, an exact replica- 
tion of thgv^indexing performed by the first lindexer in her content analysis of 
the cross validation and generalization samplles was carried out independently 
one year later. ' v 

The two sets of quantitative variables deriv^Mfrom the indexing deci- 
sions made by each indexer were analyzed "by the st(#l8e discriminant analysis 
program (BMD07M) . ® The accuracy of classification into correct criterion 
group achieved by each of t|iese two indexers was compared in order to deter- 
mine if both indexers working separately witl)- their own Individualized inter- 
pretation of the indexing rules arid conventions could achieve comparable 
classification results. The^ variables selected by the stepwise discriminant 
analysis program for the fir&t 15 steps also were compared for the two in-^ 
dexers. * ^ * 

In addition, the possibility that two itidexers sharing the indexing of 
the same data base can achieve as good classification results as either index- 
er indexing the entire data base alone was expO-ored. The way in which this 
qiiestiqri was investigated was to dufdicate the cross validation and general- 
izatioffitKched card decks containing^L the quantitative variables derived from 
the indl*lng decisions made independeritly by each indexer. .Then the two decks 
were interleaved' in order to create two new decks, each of which contained 
half of the indexing decisions of the first indexed and half of ^ the indexing 
decisions of the second indexer : Both of these "half-and-half decka also 
were analyzed by the stepwise discriminant analysis program and are referred 
to in the discussion of results as "Half and Half (E-S)" and "Half and Half 
(S-E)." In the first half-and-half deck, the experienced (E) indexer 's judg- 
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merits correspond to odd-numbered cases-^^rtille the second (jS-) indexer's judg- 
ments'correspbnd to even-numbered cases. In the second half '^and-half deck, 
the reverse is true, with t^e second indexer*s judgmeiits cor?:esponding to odd- 
titimbered cases and the experienced ind^xer^s judgments corresponding to even-^ 
numbeifed cases. ^ > ^ ' ^ ' 



Table 24 presents • a comparison of the vkril>|.^^pll€S^^ at each step in 
the stepwise disczriminant analysis and the clB:^1t|ica;titin accur achieved by 
the two indexets and the two half-and^^half; combinations of indexers using the 
original lengthy Indexing procedure to index the Evaluation Section for the S 
cross validation AT^'^. At Step 1 tKe'vatl^ble, frequency (f) of COOPEBATION, 
was selected for aH four indexers^indexer half-and-half combinations. After 
Step 1 this* unanimil^ is absent; however, certain variables in common- were ^ 
se^.ected during Step 2 tihrough Step 6 , ttamely, Total Number of 3 Welglits . ' 
(Excellent), Total Number of -1 Weights (Poor), and frequency (f> of TECHNICAL 
SKILLS. Frequency (f) of LEJADERSHIP JLND DIRECTING and f requency ' (f ) of PRO- ' 
DUCTIVITY AND ACHIEVEMENT also figure prominently as variables selected in 
common by tbie stepwise discriminant analysis process for the four indexer.s/ 
indexer half-and-half combinations. ' ' \ ' 

The lev^l of classification accuracy achieved at each step in th& step- 
wise discriminant analysis process is remarkably Similar ior the' four in- 
dexers /indexer half-and-half combinations.' A little aover 40 percent of t\ie. ' 
Cy£a$sification problem ^s solved at Step 1 regardless of which ind^xer/indexer 
combination is considered, increasing asymptotically to approximately 80 
percent at the^ step where the best classificjation performance was attained for 
each indexer /i^ndexer combination. 

* ■ . ■ ■ ' 

A similar unanimity of results was obtained in the stepwise discriminant 
analysis of the Justification Section for the cross validation AT's (see Table 
25). At Steii l(the variable. Total Number of Index Terms' Used, was selected 
by the stepwise dis'criminant analysis process for all four indexers /indexer 
hfi^lf-and-half t&Mtgl^nations. After Step 1 unanimity once again is abs^t, but 
certain key variables Wre selected in common during Step 2 through Step 6, 
namely, frequency (f) or* weighted frequency (wf). of ORGANIZATION, weighted 
frequency (wf) of SKILLS AND ABILITIES, knd weighted frequency (wf> of PRO- 
DUCTIVITY AND ACHIEVEMENT , ' " . 

. ' ■ / ■ - \ ■ /. ^ ^ 

The level of classification accutacy achieveid at each step in the step- 
wise discriminant analysis process on the Justif^ication Section also is re- 
markably similar for the^fpur indexers /indexer half-and-half combinations, > 
althbxigh the initial ievel of classification accuracy achieved is higher than 
that foif the Evaluation Section' and continues to exceed that achieved for the^ 
Evaluation Section thtr(Sbghout -the course of the stepwise discriminant analysis 
process. Approximately 63 pe§rcent of the classification problm is* solved at 
St^p 1 on the Justl^lication Section, increasing asymptotically to approximate- 
ly 95 percent at the step wl^ere the best classification performance was. at- 
tained for each :^ndexer /indexer combination^ / 
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. TABLE 24' . ' - 

COMPARISON OF THE' VARIABLES SELECTED. AT EACH STEP 
W THE STE;PWISE DISCRlMiKANT ANALYSIS AND THE 
CLASSIFICATION ilCCURACY ACHIEVED BY THE VARIOUS INDEXERS 
U^ING TIffi ORIGINAL LENGTHY INDEXING PROCEDIJRE 
CROSS VALIDATION AT's (N^138). - EVALUATION SECTION 



Variable Selected 



No. of Cro$s 
Valid. AT 
, Classified 
Correctly 



Step 1: - ' 

Experienced Indeiker (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S--E) 



f of COOPERATION 
f of COOPERATION 
£ of COOPERATION 
f of COOPERATION 



-58 
57 
58 
57 



Step 2: 

Experfanced Indi^xer /E) . 
Second Indexer )(S) 
Half and Half (E-S) 
Half a^id Half (S-E) 



Total Nymber of 3 ^Weights 
Total Number of -1 Weights 
Total Number of /-l Weights 
Total: Number of 3 Weights 



73 
68 

71 



Step 3: / 

Experienced' Indexer 
Second Indexer (S) 
Half ^ai^d Half (E-S) 
Half and Half (S-E) 



(E) 



f of TECHNICAL SKILLS 
Total Number of 3 Weights 
Total Number of 3 Weights 
Total Number of -1 Weights 



78 
77 
7« 
79 



Step 4: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
'Half and Half (S-E) 



(E) f of LEADERSHIP AND DIRECTING 
Total Number of 1 Weights 
' f of LEADERSHIP AND DIRECTING 
Total Number of 1 Weights 



85 
84 
85 
83 



\Step 5: 



Experienced Indexer 
Second Indexer (S) 
Half ah^ Half (E-S) 
Half an4 Half (S-E) 



Step 6; 



(E) 



Experleticed Indexer (E) 
Second fpdexer (S) 
Half andSHalf (E-S) 
Half and HaliE (S-E!) 



Total Number of -1 Weights 

f of PRODUCTIVITY AND ACHIEVEMENT 

f of TECHNICAL SKILLS 

wf of RESPONSIVENESS 



wf of RESPONSIVENESS 

f of TECHNICAL SKILLS^^ , ; 

f of PRODUCTIVITY ANTMACHIeVeMENT 

f of TECHNICAL SKlllS 



84 
87 

84 



82 
83 
86 
84 



(Continued) 
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' y TABLE 24 (CONT.) 

COMFieRlSON OF TRE VARIABLES SELECTED AT EACH STEP 

IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE \ 
CLASSIFICATION ACCURACY ACHIEVEtf^Y THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
CROSS VALIDATION AT' S (N=138) - EVALUATION SECTION 



Variable Selected 



No. of Cross 
Valid. AT' s 
Classified. 
Correctly 



Step 7: 

Experienced Indexer (E) 
Second Indexer (S) ■ 
.Half and Half (E-S) ' 
Half and Half (S-E) • 



f of RESOURCEFULNESS 

f of PROFESSIONALISM 

Total Number of 1 Weights 

f of LEADERSHIP AND DIRECTING 



86 

84 
84 



Step 8: 

Experienced Indexer J (E) 
.^Second Indexer (S) /" 
Half and Half (E-S) 
Half and Half (S-E) 



Step 9: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) 



f of PRODUCTIVITY' AND AQHIEVEMENT 89 

f of LEADERSHIP AND DIRECTING 85 

f of- SERVICE MOTIVATION 87 

f "of PRODUCTIVITY ACHIEVEMENT 93 

f of SERVICE MOTIVATION . 95 

wf of M/^AGEMENT FUNCTIONS 89 

wf of RELIABILITY AND DEPENDABILITY 84 

f of PROFESSIONALISM • 88 



Step 10: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E), 



(E) wf of TECHNICAL SKILLS 
f of SERVICE MOTIVATION 
wf of RESPONSIVENESS 
f of RESPONSIVENESS . 



92 
90 
85 
91 



Step 11: 

Experienced Indexer 
Second Indexer (S) 
Hali and Half (E-S) 
Half and Half (S-E) 



(E) ~ Total Number of 2 .Weights 

wf of RELIABILITY AND DEPENDABILITY 
wf of I*'k0FESSI0NALISM 
f of ORGANIZATION 



90 
89 
86 
94 



Step 12: 

Experienced Indexer (E) 
*»~--'-v^econd Indexer (S) 
H^f 'and Half (E-S) 
Half and Half (S-E) 



f of AWARDS AND PUNISHMENT 

wf .of RESPONSIVENESS 

f of RESOURCEFULNESS 

wf of INTELLECTUAL FUNCTIONING 



92 
91 
88 
95 



(Continued) 
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TABLE 24 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEt 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDUIlE 
OLbSS VALIDATION AT's (N-13S) - EVALUATION SECTION 



Step ^13: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 14: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 15: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) 



Step 20 



Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) 



(E) 



(E) 



Variable Selected 



f pf PROFESSIONALISM 

f of RESPONSIVENESS 

Sum of Variables 1 through 29 

wf of 'PLANNING 



/ 



Total NumbfiS^of Index Terms Used 
wf of COKTOOLLING 
Total Number of Index Terms Used 
f of INTELLECTUAL FUNCTIONING 



r 



Total Number of Words in Text 
wf of INTELLECTUAL FUNCTIONING 
wf of COMMUNICATION 
f of CONDUCT, INTEGRITY, AND PRIjBE 



f of ORGANIZATION 

Sum of Variables 31 through 59 

f of RESPONSIVENESS 

f of AWARDS AND' PUNISHMENT 



No. of Crijss 
Valid. AT's 
Classified 
Correctly 



92 
92 
86 
94 



93 
94 
85 
96 



94 
94 
87 
98 



100 

96 

98 T 
99 



Step 25: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) 



wf\Qf PROFESSIONALISM 
wf ^of USE OF COMMUNICATION 
wf of. ASSET TO THE NAVY 
wf of MANAGEMENT FUNCTIONS 



100 
95 
104 
105 



Best Step: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (ft-S) 
Half and Half (S-E) 



(E) f of INITIATIVE (Step 48) IIG 

f of INTELLECTUAL FUNCTIONING' (Step 58) 112 

f of FLEXIBILITY (Step 62) 110 

f of CONTROLLING (Step 54) 116 



♦ 

72 



58 



TABLE 25 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DlSCRIMIMNT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INpEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
CROSS VALIDAHION AT' S,' (N=138) - JUSTIFICATION SECTION 



Variable Selected 



No. of Cro%s 
Valid. AT '8 
Classified . 
Correctly 



Step 1: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) Total Number of Index Terms Used 
Total Number df Index Terms Used 
Total Number of liiclex Terns Used 
Total Number of Index Terms Used 



88 
86 
87 
87 



Step 2: 

Experiefnced Inde^cer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of TECHNICAL SKILLS 

wf of ORGANIZATION 

wf of TECHNICAL SKILLS 

wf of SKILLS AND ABILITIES 



97 
9A 
97 
90 



Step 3: 

Experienced Indexer 
Second Indexer (S) 
,Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of PRODUCTIVITY AIJP ACHIEVEMENT 
wf of SKILLS AND ABILITIES 
wf of PRODUCTIVITY AND ACHIEVEMENT 
f of ORGANIZATION 



98 
93 
101 

• 95 



Step 4: 

Experienced Indtxer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of SKILLS AND ABILITIES 

wf of PRODUCTIVITY AND ACHIEVEMENT 

wf of SKILLS AND ABILITIES 

wf of PLANNING 



102 
100 
101 
93 



Step 5: 

Experienced Indexer (B) 
Second Indexer (S) 
Half and Half „(E-S) 
Half and Half °(S-E) 

Step 6: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of COMMUNICATION 

f of AWARDS AND PUNISHMENT 

wf of ORGANIZATION 

f of PLANNING 



wf of ORGANIZATION 
wf of LEADERSHIP AND DIRECTING 
Total Number of -2 Weights 
f of LEADERSHIP AND DIRECTING 



101 
102 
100 
95 



100 
101 
104 
lOA 



(Continued) 
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TABLE 25 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE ' ' 
CLASSIFICATION -ACCURACY ACHIEVED BY. THE VARIOUS IlTOEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
moss VALIDATION AT' S (N=138) - JUSTIFICATIOU SECTION 



' No. of Cross 

r. ' <• . Valid. AT's 

' • - Classified 

Variable Selected Correctly 

Step 7v* ' . ^ 

Experienced Indexer (E) wf of ASSET TO THE NAVY 102 

Second Indexer (Sj f of SERVICE MQTIVATtON • 103 

Half and Half ()B-S) wf o£ AWARDS AND PUNISHl^NT 105 

Half and Half (S-E) * Sijja of Variable^ 31 through 59 . 104 



Step- 8: 



Experienced Indexer (E) ' wf of REPUTE • 104 

Second Indexer (S) d of PLANNING '101 

Half and Half (E-S) .f of REPUTE . 110 

Half and 'Half (S-E) wf of QRIVE ' . 106 

- * 

Step 9: ^ ' 

Experienced Indexer (E) wf of AWARDS AND PUNISHMENT 106 

Second Indejcer (S) • ' wf of PLANNING ^ 106 

Half and Half (E-S) . . t of SERVICE MOTIVATION 109 

Half and Half (S-E) f of AWARDS AND PUNISHMENT 110 

Step 10: — 

Experienced Inde:;er (E) . f of INTELLECTUAL FUNCTIONING 108 

Second Indexer (S) Total Number of 2 Weights 108 

Half and Half (E-S) f of MANAGEMENT FUNCTIONS , 109 

Half and -Half (S-E) wf of POTENTIAL ^ ' . 112 

Step 11: - . ♦ 

Experienced Indexer (E) wf of POTENTIAL . 109 

Second Indexer (S) * Total Number of -1 Weights 111 

Half and Half (E-S) wf of LEADERSHIP AND DIRECTING . 108 

Half and Half (S-E) wf of PROFESSIONALISM ^ - 111 

Step 12: 

Experienced Indexer (E) f of REPRESENTATION 107 

Second Indexer (S) wf of RESOURCEFULNESS 110 

Half and Half (E-S) wf of SERVICE MOTIVATION llCf 

Half and Half (S-E) wf of MANAGEMENT FUNCTIONS 109 



(Continued) 
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1 



TABLE 25 (CONT.) 



COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
-^IN THE STEPWISE DISCRIMINANT JflSALtSIS AND THE 
CLASSIFICATION ACCURACY^^AeSTEVED BY THE VARIOUS INDgXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
CROSS VMIDATION AT's (N^ISS) - JUSTIFICATION SECTION 

( 



Step 13 



Experienced Indexer (E) 
Second Indexer (S) 
Half and Half. (E-S) 
Half and Half (S-E) 



Step 14: 

Experienced Indexer 
Second Indexer (S) 
Half and Ralf (E-S) 
Half and tlalf (S-E) 



(E) 



Variable Selected 



Total Number of Words In Text 
f of REPUTE 

f of SKILLS AND ABILITIES 
f of MANAGEMENT .FUNCTIONS 



wf of RESPONSIVENESS 
Total Number of -2 Weights 
Total Number of -1 Weights 
wf of LEADERSHIP AND DIRECTING 



No. of Cross 
Valid. AT's 
Classified 
"Correctly 



108 
112 
110 
111 



Ul 
115 
110 
111 



Step 15: 

Exiferienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
ftalf and Half (S-E) 



(E) f of RESPONSIVENESS 

wf of "liANAGEMENT FUNCTIONS 
f of INITIATIVE 
, wf of ORGANIZATION 



113 
114 
112 
112 



Step 20: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 25: 

•* « 

Experienced Indexer (E) • 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) " 



f of LEADERSHIP AND DIRECTING 
■wf of SERVICE#IOTIVATION , 
f of DRIVE 

Total 'Nutab^r of 1 Weights V 



f of EfouRANCE 

Totai Number of 1 Weights 

f of LEADERSHIP AND DIRECTING 

f of ENDURANCE 



1144. 

113. 

113v 

116 



116 
116 
116 
119 



B0St Step: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of COMMUNICATION (Step 46) 129 

wf of TECHNICAL SKILLS (Step 56) 134 

Sum of Variables 31 through 59 (5tep 64) 130 

' f of COMMUNICATION (Step 63) 135 



o 

ERIC 



75 



61 




^^"^ In exariiuing^.the findings from the stepwise discriminant analysis of the 
cross validation BT*s, similar unanimity resulted. In Table 26 depicting the 
results for Jthe Evaluation Section, it can be seei\ that the variable. Total 
Number of 3 Weights (Excellent), was unanimously selected at Step X and the 
variable. Total Number of Index Terms Used, was unanimously selected at Step 
2. Other' variables that assumed early Importance on the Evaluation Section 
far the cross validation^T's were Total Number of -1 Weights (Poor) and^fre- 
quency (f) or weighted fflfquency (wf) of RESOURCEFULNESS. / 

At Step 1 of the- stepwise discriminant analysis of the Evaluation Sectipn 
for the cross validation BT's, approximately 45 percent of the classification 
problem is solved regardless of which indexer/indexer combination is consider- 
ed, increasing asymptotically to app/roximately 95 V^cent at the step where ^ 
the best classification performance was attained for each indexer/indexer 
combination. 

Paralleilfig the findings for the cross-validation AT*S^on the Justifica- 
tion Section, the cross validation BT stepwise discriminant analysis for the 
Justification Section resulted in Total Number of Index Terms Used being se- 
lected unanimously at Step 1 (gee Table 27). After Step 1 certain key vartr 
ables were selected in common from Step 2 through Step 6, namely, frequency 
(f) or weighted frequency (wf) of LEADERSHIP AND DIRECTING; Sum of Variables 1 
through 29; frequency (f) or weighted frequency <wf) of PRODUCTIVITY AND 
ACHIEVEMENT; frequency (f) or weighted freqti6ncy (wf) of CONDUCT, INTEGRITY^ 
AND PRIDE; and frequency ^/fT^olNflRIVE. 

At Step 1 on the stepwise discriminant analysis of the Justification Sec- 
tion for the cross validation BT*s, a little pver two-thirds of the classifi- 
cation problem is solved regardless of which indexer/indexer cqmbinatidn is 
considered, increasing asymptotically to unanimous perfect • classification at 
the step where the best classification performance was attained for each 
indexer/i^exer combination. 

The stepwise discriminant analysis of the Evaluation Section for the 
generalization CS*s revealed less unanimity in the variables that were se- 
lected early In the procedure (see -Table 28). However, three variables were 
prominent In the first six steps, namely, weighted frequency (wf) of CON- 
TROLLING, frequency (f) or weighted frequency (wf) of MANAGEMENT- FUNCTIONS , 
and frequency (f) or weighted frequency (wf) of SKILLS AND ABILITIES. This 
lesser unanimity in the early selection of variables may be a result of the 
relatively small size of the CS sample (N«60) compared to the other three 
occupational specialties included in this investigation. 

^ » - ^ 

Because of thiB variation in variables selected early in the stepwise 
discriminant analysis of the Evaluation Section for the generalization CS s, 
it~Sppears that classification accuracy also . experienced a variance in the 
early steps of the analysis. Classif icati6n accuracy in£o correct criterion 
group at St^p 1 ranged from one-third' to almost one-half. However, at the 
step where ^he best classification performance was achieved for each indexer/ 
indexer combination, 97 percent classification accuracy was unanimously at- 
' tained. 



/ 



62 



TABLE 26 



COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IH THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS^ JNDEXERS 
USING THE ORIGINA]^ LENGTHY INDEXING PROCEDURE 
CROSS VALIDATION BT's (N=^4) - EVALUATION SECTION 



Variable Selected 



No. di Cross 
Valid, BT*s 
Classified 
Correctly ^ 



Step 1: ^ 

\^ Experienced Indexer (E) 
Second Indexer (S) 
Half and. Half (E-S) 
Half and Half (S-E) 

Step 2: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Hal^ (E-S) 
Half and Half (S-E) 



Total Number of 3 

Tofeal Number of 3 

Total Number of 3 

Total Number of 3 



Weights 
Weights 
Weights 
Weights 



Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number of llbdex Terms Used 



39 
37 
40 
36 



52 
51 
51 
51 



1 



Step 3: ^ 

Experienced Indexer (E) 
Second Indexer (S) 
Half, and Half (E-S) 
Half and Half (S-E) 

Step 4: 

Experienced Indexer (E) 
Second Indexer (SV . 
Half and Half (E-S> 
Half and Half (S-E) 

Stop 5; 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Hair and Half (S-E) 

Stop 6: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E), 



Si^ of Va"? iables 31 through 59 
4Eotal Number of -1 Weights 
Total Number of -1 Weights 
Total Number of -1 Weights 

wf of RESOURCEFULPTESS 

wf of GROOMING AND ATTIRE 

Sum of Variables 31 through 59 

wf of GROOMING AND ATTIRE 



f of COOPERATION 
f of RESOURCEFULNESS 
wf of REPUTE I 
f of RESOURCEFULNESS 



f of RESOURCEFULNESS 
Wf of RESOURCEFULNESS 
f of REPUTE 
wf of RESOURCEFULNESS 



53 
51 
52 
52 



5: 
55 
53 
55 



57 
56 
56 
55 



58 
58 
54 
57 



(Continued) 
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TABLE 26 (CONT.) \ , 

COMPARISON 'OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 

CROSS VALIDATION BT's (tl=84) - EVALUATION SECTION 



a 



Variable Seleicted 



No. of Cross 
Valid. BT's 
Classified 
Correctly 



Step 7; 

Experience^ Indexer 
Second Indfexer (S) 
Half and Half (E-S) 
Half -and Half (S-E) 



(E) Total Number of Words in Text 
f of COOPERATION 
wf of GROOMING AND ATTIRE 
f of COOPERATION 



60 
62 
54 
58 



Step 5; 

Experienced Indexer (E) 
• Second Indexer (S) 
Half and Half (E-S) 
Half and Half j(S-E) 



f of AWARDS, AND PUNISHMENT 

f of RELIABILITY AND DEPENDABILITY 

f of INITIATIVE 

f of FLEXIBILITY 



58 
63. 
55 
59 



Step 9; 

Experienced Indexet (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) ' 

Step i^- 

Experienced Indexe'r (E) 
Second Indexer (S) 
^)q.t and Half (E-S) 
<7/Hal'f and Half (S-E) 



f of ORGANIZATION 

f of REPUTE 

f of TECHNICAL SKILLS 

f of INTELLECTUAL FUNCTIONING 



f of TECHNICAL SKILLS 

f of REPRESENTATION 

f of ASSET TO THE NAVY 

wf of INTELLECTUAL FUNCTIONING 



61 
62 

59 



6^ 
60" 
64 
61 



Step lit 

Experienced Indexer 
Second Indexer (S)' 
Half and Half (E-S) 
Half and Half (S-E) 



(E)* f of ASSET TO THE NAVY 
wf of COMMUNICATION 
wf of STAFFING 

f of RELIABILITY AND DEPENDABILITY 



62 
64 
64 
61 



Step 12: , 

Experienced -Indexer 
Second Indexer (S) 
Half and Half (E-S) 
. Half and Half (S-E) 



(E) wf of ASSET TO THE NAVY . ■ 
- f of TECHNICAL SKILLS 
f of SKILLS AND ABILITIES 
wf of RESPONSIVENESS 



64 
62 
64 
61 



(Continued) 



78 



64 



TABLE 26 (CONT.) 

COIPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY, INDEXING PROCEDURE 

CROSS VALIDATION BT's (N^84) '-^ EVALUATION SECTIOH 



4: 



Variable Selected 



No. of Cro9S 
Valid. BX'» 
Classified 
Correctly 



Step 13: 

. Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of FLEXIBILITY 

f of ASSET TO*THE NAVY 

f of RELIABILITY AND DEPENDABILITY 

f of SKILLS AND ABILITIES 



64 
65 
68 
65 



Step 14: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of REPUTE 

of ASSET TO THE NAVY 
wf of RELIABILITY AND DEPENDABILITY 
wf of PRODUCTIVITY AND ACHIEVEMENT 



62 
66 
68 
65 



$tep 15: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of STAFFING X 
wf of STAFFING 

wf of CONDUCT > INTEGRITY, AND PRID? 
wf of ORGANIZATION 



65 
67 
67 
64 



Step 20: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of PROFESSIONALISM 
f of DRIVE 

wf of AWARDS AND PUNISHMENT 
Total Number of 1 Weights 



69 
66 
70 
66 



Step 25: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



Best Step: 



Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) 



wf of DRIVE 
f of CONTROLLING 
wf of COOPERATION 
wf of DRIVE 



f of COOPERATION (Step 62) 
wf of SERVICE MOTIVATION (Step 50) 
wf of USE OF COMMUNICATION (Step 54) 
wf of CONDUCT, INTEGRITY, AND PRIDE 
(Step 62) 



71 
76 
76 
73 



82 
78 
81 
80 



ERIC 



79 
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65 



TABLE 27 

COMPARISON OF THE VARIABLES ^ELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMI^JANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTltY INDEXING PROCEDlfeE 
CROSS VALIDATION BT's (N=84) - JUSTIFICATION SECTION ' 



Variable Selected 



No. of, Cross 
Valid. BT's 
Classified 
Corredtly 



Step 1: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Ha],f and Half (S-E) 



Totdl Nuimber of Index Terms Used 
Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number of Index, Terms Used 



58 
57 
58 
57 



Step 2: * 

Experienced Indexer (E) 
Second Indexer ^S) , 
Half and Half (E-S) 
Half and Half (S-E) 



wf of LEADERSHIP AND DIRECTING 
Sum of Variables 1 through 29 
wf of LEADERSHIP AND DIRECTING 
Sum Qf Variables 1 through 29 



61 
55 
6A 
58 



Step 3: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) Sum of Variables 1 through 29 
f of LEADERSHIP AND DIRECTING 
Sum of Variables 1 'through 29 
wf of LEADERSHIP AND- DIRECTING 




Step 4: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of PRODUCTIVITY AND ACHIEVEMENT 
wf of PRODUCTIVITY AND ACHIEVEMENT 
f of PRODUCTIVITY AND ACHIEVEMENT 
. wf of PRODUCTIVITY AND ACHIEVEMENT 



61 
65 

6a 

65 



Stop is 

Experienced Indexer (E). 
Second Indexer (S) 
Half and Half (E-S> 
Half and Half (^.-E^ 

Step 6:, ( 

Experienced Indexer (E) 
Second Indexer fSO 
Half and Half (E-^) 
Half and Half (S-E) 



wf of CONDUCT,. INTEGRITY, AND PRIDE 

f of DRIVE 

f of DRIVE " 

wf of CONDUCT, INTEGRITY, AND PRIDE 



f of DRIVE 

wf of CONDUCT, INTEGRITY, AND PRIDE 
f of CONDUCT, INTEGRITY, AND PRIDE 
f of DRIVE 



63 
6A 
63 
66 



66 
64 
64 
67 



(Continued) 

80 



66 



TABLE 27 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY' THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDjEXING PROCEDURE 
CROSS VALIDATION BT' s (N=84) - JUSTIFICATION SECTION 



Variable Selected 



No. of Cross 
Valid. BT'g 
Classified 
Correctly 



Step 7; e 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-^S) 
Half and Half (S-E) 

Step 8: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
• Half and Half (S-E) 

Step 9: 

Experienced Indexer (E) 
" Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of SERVICE MOTIVATION 
f of USE OF COMMUNICATION 
f of INITIATIVE 
wf of DRIVE 



f of INITIATIVE 

wf oi ASSET TO THE NAVY 

wf of REPUTE 

f of RESPONSIVENESS ' 



f of RESPONSIVENESS 
wf of MANAiSEMENT FUNCTIONS 
wf of ASSET TO THE NAVY 
wf of ENDURANCE 



6*6 
67 
70 
67 



72 
68 
73 
70 



74 
72 
76 
70 



Step 10: 

Experienced Indexer 
. Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of ASSET TO THE NAVY 
wf of REPUTE 

f of- MANAGEMENT FUNCTIONS 

wf of INTELLECTUAL FUNCTIONING 



71 
72 
74 
70 



Stop 11: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half arid Half (S-E) 



(E) wf of DRIVE 

f of ORGANIZATION . 
wf of RESOURCEFULNESS 
f of COMMUNICATION 



75 
72 
74 
70 



Step 12: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of RES^CEFULNESS 

wf of INTELLECTUAL FUNCTIONING 
f of REPUTE 

wf of ASSET TO THE NAVY 



78 
- 69 
74 
73 



N 



ERIC 



< Continued) ^ 



81 



\ 

TABLE 27 (GONT.) 

COMPARISON OF THE VARIABliES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXEkS 
r USING THE ORIGINAL LENGTffif INDEXING PROCEDURE 
CROSS VALIDATION BT's (N~84) JUSTIFICATION SECTION 



Variable Sblected 



No. of Cross 
Valid. BT's 
Classified 
Correctly 



Step 13: 

Experienced' Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of RESOURCEFULNESS 
wf of STAFFING 
Total Number of 2 Weights 
f of FLEXIBILITY 



76 
74 
74 
73 



Step 14: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S*E) 



f* of PROFESSIONALISM 

v,f of Staffing 
wf of staffing 

wf of' FLEXIBILITY 



77 
73 
74 
75 



Step 15: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 




f -of REPRESENTATION 
Total Number of 3 Weights 
f of ENDURANCE 
wf of COMMUNICATION 



76 
75 
73 
74 



Step 20: ^ 

. Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of REPUTE 

wf of ENDURANCE. 

f of TECHNICAL ^ILLS 

f of INITIATIVE 



79 
80 
78 
77 



Stop 25: 

Expe/ienccd Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of PLANNING 

f of ASSET TO THE NAVY 

wf of MANAGEMENT FUNCTIONS 

f of LEADERSHIP AND DIRECTING 



80 
84 
81 
80 



Bast Stop: 

Experience!^ Indexer (E) 

Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of RELIABILITY AND DEPENDABILITY 84 

(Stop 46) \ 

f of ASSET TO THE 1*AVY (Step 2$) 84 

wf of SKILLS AND ABttlTIES (Stop 60) 84 

f of RESOURCEFULNESS (Stop 36) , • 84 



82 



68 



TABLE 28 ' 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
. JN THE STEPWISE DISCRXMINANT AEIALYSIS AND THE* 
CLASSIFICATION ACCURACY ACHIE^d' "BY THE VARIOUS^ IHDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
GENERALIZATION CS's (N=6g) - EVALUATION SECHlON 



Step 1: 

fekperienced Indexer (e) 
• Second Indexer (S) 
Half and Half (E-S^ 
^ Half and Half (S-E) 



Variable Selected 



wf of CONTROLLING 

M;rf oi RESPONSIVENESS 

wf of l^ACaiENT -idNCTIONS 

wf of CONTRpLLINfe^-'' 



No, of Gener- 
alizat*n OS's 
Classified 
Correctly; 



20 
27/ 
28 
20 



St ejp 2: 

Experienced Indexer (E) 
Second Indexer (S) 
'Half arta Half (E-S) 
Half and Half (S-E) 

Step 3: 

tepcperi^nced Indexer (E) 
Second Indexer (S) 
'Half and Half (E-S) 
Half and Half (S-E) 

Step 4: , 

Experienced Indexer (E) 
Second Indexer (S) 
, HaM and U^f (E-S) 
Half and Half , < S-E) 



wf of ASSET TO, THE NAVY 
wf of CONTROLLING . 
wf of ASSET TO THE NAVY 
wf of RESPONSIVENESS" 



f of MANAGEMENT FUNCTIONS 
wf of SKILLS AND ABILITIES 
wf of SKILLS AND ABILITIES 
wf of SERVICE MOTIVATION 



r 



wf of SKILLS AND ABILITIES 
Total Number of Words In Text 
To-Cal Number of Index T^rms Usfed 
Total Numbei: of 1 Weights , ' 



28 
29 
30 
2& 



29 
30 



30 



30 
34 

31' 
31 



?tep ,5: , 

Ekpetienced Indexer (E) 
fiecond Indexer : (S) 
Half - and" Half\(E-S ) 
Half and Half (S-E) 




lexer 

tx (s) 

(E^S) 

^Half and Half (S-E) 



(E) 



Total Number of 'Index T,erms Usi^d 
f of MANAGEMENT FUNCTKJNS / ■ 
w€- of ENDURANCE . ' / • 

f Of CONDUCT, INTEGRITY, AND PRIDE 

f of ENDURANCE / • , , /* 

wf of CONDUCT, INTEGRITY, AND PRIDE 
f of asset" TO T^E/NAVY • . 
,f of SKILLS AND ABILITIES 




32 
33 
33 
32 



34 
34 
36 
29' 



(Cpntinued) 



TABLE 28 (CONT.) 



COMPARISON OF THE VARIABLE? SELECTED ST EACH STEP 
li THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS. INDEXERS 
y USING THE ORIGINAL LENGTHY INDUING PROCEDURE 
GENERALIZATION CS's -(N^60) - EVALUATION SECTION 



f 



Variable Selected 



No. of Gener- 
al^izat'n CS's 
Classified 
Corrfectly 



Step 7; 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
"Half aud HalT (S-E) 

Step B: 

V 

Experience4 Indexer (E) 
Second Indexer (S) 
Half and' Hal'f (E-S5 
Half and Half (^-E) 

Step* 9: ' ' 

Experiemcfed Indexer (E) 
Second Indexer (S) 
Half an,d Ualf (E-S) 
Half arid Half (S-E) 

Step 10:. 

Experienced Indexer (E) 
Secottfi Indexer (S) 
Half%nd Half (E-S) 
• Half and Half (S-E)^ 



Sum of ^Variabl^ 31 through 59 
f of PLANNIJlG 
f of PLANNING 

Total Number of Words in Text 



wf of ORGANIZATION 
f of ENDURANCE 
*^wf of CONDUCT, INTEGRITY, AND PRIDE 
wfof CONDUCT, INTEGRITY, AND PRIDE 



,wf of POTENTIAL ' 
't'^f COMMUNICATION 
f of RESOURCEFULNESS 
wf .of INITIATIVE 



wf of PLANNING 

f of GROOMING MD ATTIRE 

f of CONDUCT, INTEGRITY, AND PRJDE 

f of COMMUNICATION 



38^ 
37 
41 
34 



39 
39 
40. 
•38 



4 

A2 
41 

- 39 



43 
41 
'42 
»4'0 



Step 11: i 

Experienced Indexer 
Safcond Indexer (S) 
Hlif ^nd Half (E-S) 
. Waif and'Ha,lfe (S-E) 



(E) f of SERVICE MOTIVATION- 
wf of INITIATIVE 
To t;^ J, Number of 3 Weights 
- , f of ^MAIJAGEMENT FUNCTIONS 



46 
43 

44 
39 



Step 12-: ' ' ■ . 

Experienceal Indexer 
; Second Indexer^ (S) 
Half a'nd Half (E-S) 
lialf ,and Half Cs-E)^ 



(E) ■ f of PLANNING 

. . f of C9OPERATION 
wf-'of POTENTIAL 
f of I^ANNING 



41 
45 
49 
41 



(Continued) 



ERIC 



TABLE 28 <CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT flACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS 'aND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING 1:HE original lilNGTHY INDEXING PROCEDURE 
GENERALIZATIOU CS's (N^60) - EVALtJATION SECTION 



Variable Selected 



No. of Gener- 
alizat'n CS's 
Classified 
Correctly 



Step 13: r 

Experience^ Indexer, (E) 
Secot^ Indexer (S) 
Half and Half (E-S) 
Half and Half *(S-E) . 



total Numbe-r of 1 Weights 

wf of RELIABILITY AND DEPENDABILITY 

f of RESPONSIVENjESS 

f of ENDURANCE 



44 
43 
46 
44 



Step 14: ^ . 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 

Halt and Half (S-E) , 

flv ■ ■ 

step 25c 

Experienced Indexer (E) 
Second Indexer (S) " 
.Half and Half (E-S) 
Half and: Half (S-E) 



f of LEADERSHIP AND DIRECTING 
Total Number of 1 Weights . 
wf ^of RELIABILITY AND DEPENDABILITY 
f ot GROOMING AND ATTIRE 



f of INITIATIVE 

.wf of PRODUCTIVITY AND ACHIEVEMENT 
f of RELIABILITY AND DEPENDABILITY 
wf of PRODUCTIVITY AND ACHIEVEMENT 



43 
43 
48 
46 



44 
47 
48 
46 



Step 20: 

Experienced Indexer 
Second Indexer (S) » 
Half and Half (E-S) 
Half and* Half <S-E) 



(E) f of INTELLECTUAL FUNCTIONING 

. f of RELIABILITY AND DEPENDABryl3 
- Total Number of 2 Weights ' 



wf of CQOPERATION . 



50 
49 
48 
47 



Step 25: 

Experienced Indexer (E) 
Second Ind^exei5j (S) ^ 
Half and Half (E-S) 
Hal4 and Half (S-^l) 



wf .of COOPERATION 
w£j of DRIVE , 
wf of DRIVE 
wf of POTENtlAL' 



,51 
/ 52 
^49 



Best Step: 

Experienced Indexer -(E) 
Second Indexer (S) 
%alf and flalf (E-S) 
Half arid Half (S-E) 



f of COMMUNICATION (Step 44) 
wf pf RESOURCEFULNESS, (Step 52^ 
f of POTENTIAL (Step £2) 
w£ of ORGANIZATION (^tep 54) 



58 , 

58 

58 



71 



Unanimity in the early s^l^ctiotj of variables oncie again comes into play 
in examining the results of the stepwise discriminant analysis of the general- 
ization CS's on the Justification Section , (see Table 29). At Step 1 Total 
Number of Index Terms Used was th^e variable selected for all four indexers/ 
indexer half-and-half combinations. At. Step 2 frequency (£) of PROFESSIONAL- 
ISM was unanimously Selected. In the next few. steps, weighted frequency (wf) 
of CbMMUNigATION and weighted f requency (wf ) of REPRESENTATION are prominent*. - 

AppraxiWtely two-thirds of the generalization CS's wete classified into 
correct criterion group at Step 1 for the Justification Section^ regardless of 
which indexer/indexer combination is considered. At the; fftep where the best 
classification perf ortnanceo was attained'for each indexer/indexer combination, 
perfect classification was achieved unanimously. 

On the Evaluation Section for the generalization RM's, quite remarkable 
unanimity was achieved in the selection of variables at the first three steps 
in the stepwise discriminant analysis procedure (see Table 30). Total Number 
of -1 Weights (Poor) was selected unanimously at Step 1. At Step 2 weighted 
frequency (wf) of AWARDS AND PUNISHMENT was selected unanimously, and weighted 
frequency (wf) of POTENTIAL was the variable unanimously selected at Step 3. 
Total Number of 3 Weights (Excellent) and frequency (f) of RELIABILITY; AND 

DEPENDABILITY also were prominent in the early selection of variables. 

ft - ^ 

Forty-three percent of the generalization RM's were classified correctly 
on the Evaluation Section at Step 1^; all four iridexers/indexer .half-and-half 
combinations being identical in this regard. Classification performance con- 
tinued to improve asymptotically \to a maximum? ot Approximately 80 percent at 
the step where the best classification, performance was attained for each in- 
dexer/indexer combination. ^ 

^ V . \ 

Remarkable /unanimity also was achievjed in t^e early selection of vari- 
ables in the^^i^pwise discriminant analysis of the gejneraliz^tion RM's on the 
Justification Section (see Table 31)^ Total Number of Indllpx Terms Used was 
selected unanimously at Step 1. At Step 2 Sum^'of Variables 31 through 59 was 
the variable unanimously selected, and frequency (f) or weighted frequency 
(wf ) of PRODUCTIVITY AND ACHIEVEMENT was selected unanimously at either Step 3 
or Step 4. Other va'tiables figuring prominently in early selection by the ^ 
stepwise discriminant analysis procedure were frequency (f) of COOPERATION, 
frequency (f) or weighted frequency (wf) of ENDURANCE, and weighted frequency 
(wf ) of GROOMING AND ATTIRE. 

Apprpxima'tely 65 percent of the generalization RM's were classified 
correctly on the Justification Section at Step 1, regardless of which indexer/ 
indexer combination Is considered. Classification performance continued to 
^improve asymptotically to a maximum of approximately 90 percent at the step 
Where the best classification performance was attained for each indexer/in- 
dexer combination. 

It is quite clear from these results that the two Indexers who indepen- 
dently^ indexed the cross validation and generalization samples using the 
lengthy indexing procedjare arrived a^ very similar indexing decisions, as^ 



8G 
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TABLE 29 

COMPARISON OF THE VMIIABLES SELECTTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
.CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORI(ilNAL LENGTHY INDEXING PROCEDURE 
GENERALIZATION CS's (N^SO) -JUSTIFICATION SECTION 



Variable Selected 



No. of Gener- 
alizat^n CS^s 
Classified 
Correctly 



Step 1: 

Experienced Indexer *(E) 
. Second Ind ejcer . ( S ) 
Half. and Half (E-S) 
Half and Half (S-E) 



Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number bf Index Terms Used 



42 
39 
39 
41 



Step 2: 

Experienced Indexer 
Second Indexer (S) 
' Half and Half (E-S) 
Half and Half (S-E) 



(E) 



f of PROFESSIONALISM 

f'of PROFESSIONALISM 

f of PROFESSIONALISM 

f of PROFESSIONALISM 



44 
39 
39 

.44 



Step 3: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



Step 4: 

Experienced Indexer 
Second Indexer (S) 
Half and .Half (E-S) 
Half and Half (S-E) 



(E) 



f of INITIATIVE 
wf of COMMUNICATION 
wf of COMMUNICATION 
f of INITIATIVE 

wf of COMMUNICATION 
wf of REPRESENTATION 
wf of REPRESENTATION 
wf of COMMUNICATION 



44 
45 
45 
43 



47 
46 
46 
47 



Step 5: 

Experienced Indexer 
Second' Indexer (S). ' 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of REPRESENTATION 

w'f of MANAGEMENT FUNCTIONS 
•f of SKILLS "AND ABILITIES 
wf of REPRESENTATION 



47 
48 
47 
47 



Step 6: 

Experienced Indexer (E) 
Secon4 Indexer (S) 
Half, and Half (E-S) 
Half and Half (S-E), 



wf of COOPERATION 

wfof TECHNICAL SKILLS 

f of TECHNICAL SKILLS 

wf of LEADERSHIP AND DIRECTING 



47 
50 
47 
47 



(Continued) 
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TABLE 29 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT 'ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE* 
GENERALIZATION CS's (N-60) - JUSTIFICATION SECTION 



Variable Selected 



No. of Gener- 
alizat'n CS's 
Classified 
Correctly 



Step 7: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-:E) . 



f of CONTROLLING' , 

wf of PJIODUCTIVITY ANffJ ACHIEVEMENT 

wf of INITIATIVE 

f of MANAGEMENT FUNCTIONS 



50 
52 
^ 48 
49 



Step 8: • 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
H^lf and Half (S-E) 



(E) f of POTENTIAL 

f of ASSET TO THE NAVY 

f of PRODUCTIVITY AND ACHIEVEMENT 

wf of COOPERATION 



51 
52 
50 

49: 



Step 9: 

Experienced Indexer 
Second Indexer (S), 
Half and Half (E-S) 
Half and Half (S-E) 



(E) ^f of PRODUCTIVITY AND ACHIEVEMENT 
wf of USE OF COMMUNICATION 
Total Number of 1 Weights 
f of RESPONSIVENESS 



55 
53 
55 
52 



Step 10: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of SKILLS AND ABILITIES 

^ Of SKILLS AND ABILITIES 

wf of PRODUCTIVITY AND ACHIEVEMENT 

wf of PRODUCTIVITY AND ACHIEVEMENT 



53 
52 
55 

53- 



Step 11: \ 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
' Half and Half (S-E) 



(E) wf of PLANNING 

wf of INITIATIVE 

wf of USE OF COMMUNICATION^ 

Total Number of 1 Weights 



-54 

5V 
55 



Step 12: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half . (S-E) 



Total Number of Words in Text 
f of TECHNICAL SKILI^ 
wf of MANAGEMENT FUNCTIONS 
wf of INITIATIVE 



54 
53 
56 
56 



(Continued) 



TABLE 29 (CONT.) 



' COMPARISON OF THE VARIABLES SELECTED AT EACH" STEP. 
IN THE SSCEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATI€«i ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURiB 
. GENERALIZATION CS"»S (N=60) - JUSTIFICATION SECTION 



Step 13: 

Experienced Indexer *(E) 
Second Indexer^ (S) 
Half and Half (E-S) 
Half and Half (S-E) 



Variable Selected 



g^f of PROFESSIONALISM' 
w^of RESOURCEFULNESS 
wf of TECHNICAL SKILLS 
f of REPRESENTATION 



No. of Gener- 
alizat'n CS's 
Classified 
Correctly 



55 
53 
57 
56 



Step 14: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of SKILLS AND ABILITIES 

f of PRODUCTIVITY AND ACHIEVEMENT 
f of DRIVE 
/ wf of POTENTIAL 



56 
55 
56 
55 



Step 15: 

Experienced Indexer 
Second Ind^er (S) 
Half and Hajf (E-S) 
Half and Half (S-E) 



(E) f of RELIABILITY AND DEPENDABILITY 

f o£ DRIVE 

f of REPRESENTATION 

f of PRODUCTIVITY AND ACHIEVEMENT 



56 
57 
57 
52 



Step 20: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of SERVnCE MOTIVATiON. 
f of POTENTIAL 
wf of SERVICE MOTIVATION 
wf of RESPONSIVENESS 



58 

57 
58 
55 



Step 25: 

tecperlenced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Best Step: 

Experienced Indexer (E) 
Second Indexer (S) 
Half atid Half (E-S) 
Half and Half (S-E) 



wf of STAFFING "58 

Total Number of Words 1^ Text 60 

f of COMMUNICATION "60 

wf • of TECHNICAL SKILLS 59 



wf of POTENTIAL iStep 35) 6a 

Total Nuiftber of Words In Text (.Step 25) 60 

f of COMMUNICATION {Step 25) 60 

wf of MANAGEMENT FUNCTIONS (Step 40) 60 



89 



-75 



TABLE 30 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INCEXERS 
USING THE ORIGINAL LENGTHY ' INDEXING PROCEDURE 
GEmRALIZATION RM's (N=162} - Efi^UATION SECTION 



"V^Step ^; 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 2: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half, and Half (S-E) 

Step 3: ' 

Experienced Indexer (E) 
Second Indexer (S) 
Hair and Half (E-S) 
Half and Half (S-E) 



/ardable Selected 



Total Number of ^1 Weights 

Total Number of -1 Weights 

Total Number of -1 Weights 

Total Number of -1 Weights 



wf of AWARDS AND PUNISHMENT 
wf of AWARDS AND PUNISHMENT 
wf of AWARDS AND PUNISHMENT 
wf of AWARDS AND PUNISHMENT 



wf of POTE^JTIAL 
wt o f POTENTIAL 
•wf of POTENTIAL 
wf of POTENTIAL/ 



No, of Gener- 
alizat'n RM's 
Classified 
Correctly 



/ 



70 
70 
70 
70 



79 
80 
80 
79 



87, 
91 
85 
87 



Step 4: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of MANAGEMENT FUNCTIONS 
f 6f RESPONSIVENESS 
f of RESPONSIVENESS 
Total Number of 3 Weights 



88 
93 
87 
91 



Step 5: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) Tota'l Number of 3 Weigh,t8 
Total Number of 3 Weights 
wf of MANAGEMENT- 'FUNCTIONS 
f of RELIABILITY AND DEPENDABILITY 



91 
97 
93 
93 



Step 6: 

Experienced Indexer 
Seconji Indexer (S) 
Half and Half (E-S) 
' Half and Half (S-E) 



(E) f of RELIABILITY AND DEPENDABILITY, 

f of RELIABILITY AND DEPENDABILITY 

f of MANAGEMENT FUNCTIONS / 

f of FLEXIBILITY 



100 
103 
9A 
93 



ERIC 



r 



(Continued) 
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TABLE 30 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
.GENERALIZATION RH's (N=162) - EVALUATION SECTION 



Step 7: 

Experienced- Indexer (E) 
Second Indexer (S) 
Half and Half (E-S)- 
Half and Half (S-E) 



VarjLa 



Va rjable Selected 



f of COJlMUNI CATION 

wf of CONTROlilNG 

f of RELIABILITY AND DEPENDABILITY 

wf of CONTROLLING 



No. of Gener- 
alizat'n RM's 
Classified 
Correctly 

10^ 
9^ 
96 
95 



Step 8: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of POTENTIAL 
wf of REPUTE 

Total Number of 3 Weights 
wf of REPUTE 



99 
98 
100 
99 



Step 9: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (^S-E) 

Step 10: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of REPUTE 
wf of ENDURANCE 
wf of REPUTE 
wf of ENDURANCE 

f of R^UTE 

Total Number of 2 Weights 
wf of SKILLS AND ABILITIES 
Total Number of Words in Text 



95 
99 
102 
98 



94 
105 
103 ( 

96 



Step 11: 

• Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) f of INTELLEQTUAL FUNCTIONING 
f of INITIATIVE 
f of ORGMIZATION 
wf of MANAGEMENT FUNCTIONS i' 



92 
109 
104 
100 



Step 12: 

3 Experienced Indexer 
Second- Indexer (S) 
Half and Half (E-S) 
Half and Jlalf- (S-E) 



(E) f of RESPONSIVENESS 

wf of MANAGEMENT FUNCTIONS 
,f of POTENTIAL 
f of REPRESENTATION 



98 
107, 
106 
103 ' 



(Continued) 
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TABLE 30 CCONT.) 



COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
GENERALIZATION RM's (N=162) - EVALUATION SECTION 



Variable Selected 



No. of Gener- 
alizat'n RM's 
Classified 
Correctly 



Step 13: 

Experienced Indexer 
Second Indexer (S) 
. Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of RESPONSIVENESS 
f o'f CONTRaLLING 
wf of CpNTROLLING 
wf of STAFFING 



99 
109 
111 
102 



Step .14: 

Experienced Indexer 
Second Indexer (SO 
Half and Half (E-S) 
Half and Half (S-E) 



(E) ' wf of SERVICE MOTIVATION 
wf of DRIVE 
f of PUNNING 
f of INITIATIVE 



99 
106 
112 
104 



Step 15: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
_ Half and Half (S-E) 

Step ^0: 

Experienced Indexer (E) 
Second Indexer (S) 

Half and Half (E-S) 

Half arid Half (S-E) 



wf of PRODUCTIVITY AND ACHIEVEMENT 
f of MANAGEMENT FUNCTIONS 
wf o-f SERVICE MOTIVATION 
f of S^VICE MOTIVATION 



wf of PLANNING 
f of FLEXIBILITY 
f of REPRESENTATION 
wf oi DRIVE 



105 
109 
113 
106 



107 
115 
115 
114 



Step 25: 

Experienced Indexer 
Second ^ Indexer ( S ) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) wf of SKILLS AND ABILITIES 
f of REPUTE 
w£ of COMMUNICATION 
wf of PROFESSIONALISM 



113 
122 
115 
122 



Best Step: 

Experienced Indexer (E) 

Second. Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



f of INTELLECTUAL FUNCTIONING (Removed) 
{Step 63) 

Sup of Variables 31 through 59 (Step 4^) 
f of GROOMING AND ATTIRE {Step 60) 
wf of SKIliLS AND ABILITIES (Step 56) 



131 

128 
130 
133 



ERIC 



92 
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TABLE 31 ,4 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
"CLASSIFICATION ACCURACY AtHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL tENGTHY INDEXING PROCEDURE 
GENERALIZATION RM's (N='162) - JUSTIFICATION SECTION' 



Step 1: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S)' 
Half and Half (S-E) 



Variable Selected 



Total Number of Index Terms Us.ed 
Total Number of Index Terms Used 
Total Number of Index Terms Used 
Total Number of Index Terms Used 



No. of Gener- 
alizat'n RM's 
Classified 
Correctly 



103 
104. 
105 
102 



Step 2: 

Experienced Indexer 
Second Indexer (S) 
Balf and Half (E-S) 
Half and Half (S-E) 

Step 3: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



(E) Sum of Variables 31 through 59 
Sum of Variables 31 through 39 
Sum of Variables 31 through 59 
Sum of Variables 31 through 59 



(E) wf of PRODUCTIVITY AND ACHIEVEMENT 
wf of PRODUCTIVITY AND ACHIEVEMENT 
f of PRODUCTIVITY AND ACHIEVEMENT 
wf of PRODUCTIVITY AND ACHIEVEMENT 



105 
105 
106 
104 



110 
114 
110 
113 



Step\ 4: 

Experienced Indexer (B) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 5; 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 

Step 6: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) ' 
Half and Half (S-E) 



f of PRODUCTIVITY AND '.ACHIEVEMENT 
f of PRODUCTIVITY AND ACHIEVEMEHT 
wf of PRODUCTIVITY AND ACHIEVEMENT 
f of PRODUCTIVITY AND ACHIEV^NT 



f of C©OPERATION 
f of ENDURA^qCE, 
f of COOPERATION 
f of COOPERATION 



wf of GROOMING AND ATTIRE 
wf of GROOMING 'aND ATTIRE 
f of CONTROLLING 
wf of GROOMING AND ATTIRP 



114 
117 
113 
117 



117 
122 
120 
119 



123 

123 
122 
125 



(Continued) 
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.TABLE 31 (CONT.) 

COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 
IN THE STEPWISE DISCRIMINANT ANALYSIS AND THE 
CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
GENERALIZATION RM's (N=162) - JUSTIFICATION SECTION 



Variable Selected 



No. of Gener- 
alizat'n RM's 
Classified 
Correctly 



Step 7: 

Experienced Indexer 
Second Indexer (S) 
Half and Half (E|^) 
Half and Half (sIe) 



(E) wf of STAFFING 

f of COOPERATION 

Total Number of I Weights 

f of ENDURANCE 



127 
123 
125 
125 



Step 0: 

Experienced Indexer 
Second Indexer (S)' 
Half and Half (E-S) 
- Half' and Half .(S-E) 



(E) wf of ENDURANCE 
f of CONTROLLING 
"wf of RESOURCEFULNESS 
wf pf . PLANNING 



130 
.125 
127 
125 



Step 9: 

Experienced Inde^ier 
Second Indexer (S) 
. Half and Half (E-S) 
' Half and Half (S-E) 



(E) •° f.of STAFFING 

f of PLANNING 

f of TECHNICAL SKILLS 

f of STAFFING 



131 
126 
130 
124 



Step 10: 

* Experienced Indexer (E) 
Second Indfexer (S) 
Half and Half (E-S) 
Half and Hdlf (S-E) 

Step 11: 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) 
Half and Half (S-E) 



wf of PLANNING 
wf of RESOURCEFULNESS 
f of RESOURCEFULNESS 
wf of STAFFING 



Total Number of 2 Weights 
f of RESOURCEFULNESS 
f of POTENTIAL 

Total Number of Words In Text 



134 
132 
131 
126[ 



129 
132 
130 
133 



Step 12: 

Experienced Indexer 
Second Indexer (S) 
Half and .Half (E-S) 
Half and Half (S-E) 



(E) f of POTENTIAL 

wf of CONTROLLING 
f of REPUTE 

wf of MANAGEMENT FUNCTIONS 



130 
131 
129 
131 



ERIC 



^CtiChtinuGd) 
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. ' TABLE 31 <CONT.) ' • *. 

V COMPARISON OF THE VARIABLES SELECTED AT EACH STEP 

- IH THE STEPWISE DISCRIMINANT ANALYSIS AND THE 

CLASSIFICATION ACCURACY ACHIEVED BY THE VARIOUS INDEXERS 
USING THE ORIGINAL LENGTHY INDEXING PROCEDURE 
GENERAJLIZATION JRM's (N'162) - JUSTIFICATION SECTIOU 
!} • ■ ■ . ■ • • • , > 

. . No. of Gener- 
alizajt'n RM's 



Classified 

Variable Selected ■ . Correctly 



Step 13: ' 

Experienced- Indexer (E) f of INITIATIVE . 130 

Second Indexes >(S) , f of, COMMUNICATION 132' 

Half and Halt (E-S) wf of STAFFING 129 

' Half and Half (S-E) f MANAGEMENT FUNCTIONS , 131 

Step I4i 

Experienced Indexer (E) wf of TECHNICAL SKILLS * 133 

Second Indexer (*) f of REPUTE * ^ . 134 

Half and Half (E-S) wf of GROOMING AND ATTIRE ' '127 

Half and Half (S-E) f of COMMlliJICATIQN ," 132 

Step IS: 

Experienced Indexer (E). ^ f of REPUTE 132 

Second Indexer (S) wf of COMMUNICATION " 134 

Hdlf and Half (E-S) wf of CONTROlJcNG 127 

Half and Half^ (S-E) .' f of RESPONSWENESS 134 

\ 

Step 20 t 

Experienced Indexer (E) wf of PROFESSIONALISM 142 

Second Indexer (S) ' f of LEADERSHIP AND DIRECTING 135 

Half and Half (E-S) f of INITIATIVE 133- 

. Half and Half (S-E) f of POT^TIAL 135 

Step 25t 

Experienced Indexer (E) wf of USE OF COMMUNICATION^ 136 

Second Indexer (S) f oi ASSET TO THE NAVY 136 

Half and Half (E-S) wf of ENDURANCE * * 136 

Half and Half (S-E) ^ wf of INITIATIVE * 138 

Best Stopt ■ ' . 

Exper/£enced Indexer (E) wf of FLEXIBILITY (Stop 40} 144 

Secorid Indexer '(S) f of' POTENTIAL (Stop 38) 141 

Half and Half (E-S) f GROOMING AIJD^ATTIRE (Step 58) 148 

Half and Half (8-E) f of TECHNICAL SKli:tSv (Step 44) 140 
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reflected In the unanimity of variables selected early by the stepwise dis- 
criminant analysis procedure ^nd in the similar classification accuracy 
achieved at each step in the stepwise discriminant analysis process.. For the 
four occupational specialties represertted In the cross validation and general- 
ization samples, no variable in common for all four indexers/indexer half-and- 
half combinations was selected early on the Evaluation Section. However two. 
variables were selected early for three of the -four Indexers/indexer half-and- 
half combinations, namely. Total Number of, 3 Weights (Excellent) and Total 
Number of -1 Weights (Poor). This finding suggests that on the Evaluation 
Section the acijectiVes and adverba that an evaluator uses to describe the 
performance of an individual that Is being evaluated are the most , important 
characteristics of the narrative text for differentiating between superlative 
chief petty officers and theit slightly less qualified colleagues. Other key 
variables are specific to each of the fouV occupational specialties and 
corroborate the findings in earlier studies.* ^ ^ 

On the Justification Section the variable unanimously seli5;cted at Step 1. 
for bll four occupational specialties represented in -the cross validation and 
Eeneralization samples was Total Number of Index Xerms Used. This variable, 
without exception also was selected first for the "^"f ^"'^i£|» JlJ- 

earlier studies of these same four occupational specialties.^ This ^able 
reflects the variety of specific areas of an individual's performance- that an 
evaluator chose to comment on, and is measured by the number of different 
index terms dhosen by the indexer to encompasb the narrative content. The 
initial choice of this variable for the Juatif icat ion Section alBq is a re- 
flection of the fact that the narrative comments on thf Justification Section 
tvDically are longer than thbse on the Evaluation' Section. This finding 
indicated once again that the range" of skills and abilities that a chief pptty. 
officer manifests is a key factor in his superior pe^rformance as n-arrated by 
the evaluator in the Justification Section. Productivity and Achievement also 
was a variable selected early on the justification Section for three of the 
four indexers/indexer half-arid-hajlf combinations. Other key variables are 
specific to each of the f ouvwoccupatiowil specialties and parallel the results 
in eaflier studies.^ 

A recapitulation of the classification accuracy achieved by the four in- 
dexers/indexer half-and-half combiflatiqP^ using the original lengthy indexing 
froceduro is provided in Table 32. The similarity of clasai icat- on accuracy 
achieved la striking for all four occupational specialties studied onj^th the 
Evaluation Section and the Justification Section. Without exception better 
classification was achieved in the content analysis of the narrative comments 
in the Justification Section compared to the Evaluation Section, "^hese re- 
sults replicate the findings in earlier studies of these same four occupa- 
tional specialties.^ ' ' , ■ 

The conclusidn that can be drawn from this investigation is that regard- 
less of which -fndivi'dualized interpretation of the indexing rujes and conven- 
tions that may be used to index a particular data base of narrat ve perfor- 
mance eval^tiona, a similar, level of claosif ication agreement with the Xri- 
terion of on-job performance can be expected. Fur therraiofrp , it can be con- 
eluded that two indexert^ sharing the indexing of the^ame data base can be 
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TABLE 32 



RECAPITULATION OF THE CLASSIEICATION^, ACGURACY 

' . ACHI|^D BY. THE VARIOUS. INDEXERS' 
USING THE (»iGlNAL LENGTHY -iilDEXING PROCEDURE* 



r Sample • 

CROSS VALIDATION AT ' S 
Evaluation Section 



Justification Section 



CROSS VALIDATldN BT's 
Evaluajiion "iSection 



.Justification ^ectlon 



GENERALIZATION ds' S ' 
Evaluation Section 



Justification S^ct?±on 



Indexer 



Classification 
] Accuracy 



Experienced Indexer (E) 
Second Indexer (SJ 
Half ai^d Half (E-S) 
Half atd Half (S-E) 

Experienced Indexer "(E) 
Second Indexer (S) 
Half ai^id Half (E-S) ' 
.Half and Half (S-E) 

Experienced Ind^er (E) 
Second Indexer (S) 
Halt and Half (E-S/ 
Half a*d galf (S-E)^, ^ 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half. (E-S) 
Hdlf and Half (S-E) 



Experienced Indexer (E) 
Second Inclexer- (S) 
Hklf and'^Hflf (E-S) 
Half and Half (S-E) 

Experienced Indexer (E) 
Second Indexer (S) 
Half and Half (E-S) , 
Half and Half (S-E) 



•lib ou^ of^ 
112 out of 
110 out^f 
116 but of 

129 out of 

134 out of 

130 out of 

135 out of 

.J ' - 



138 (80%) 

138 (81%) 

138 (80S:) 

138 .(84%) 

138 (93%) 

138 (97%) 

138 (94%) 

138 (98%) 



78 out\of 
81 fcut dsr 



82 but of 84 (98^)' 
84 (93%T. 
'84 (96%) 
.80 out of 84vjCi5%) 

84-^100%) 
84 (100%) 
84 (ioq%) 
84 \lOO%)" 



84 out of 
84 out ^ 
.84 out^ot 
84 out of 

7 ' 



\58 out of 60 (9|g) 

58 out of 60 (97%) 

, . 58 out of -60 (97%) 

« 58 but of 60 (97%) 

6Qr out o f, 60 C100%) 

6Q out of 60- (100%) 

60 out of 60 (100%) 

' 60 out pf 60 (100%) 



GENERALIZATION RM's' 
Evaluation Section 



Justification Section 



Experienced* Indexpr 
Second Indexer' (S) 
Hdlf and Half (E-S) 
Half and Half , (S-E)' 

Experienced Indexer 
Second Indexer (S) 
Half ana Half (Er-S) 
Half and Half (S-E) 



if' 

(E) 131 out of 162 (81%) 

128-out of 162 (79%) 

130 ojit of 162 (80%) 

.133 Out iDftl62 (82%) 

(E) -< 144 out of 162 (89%) 

■ 141 out qf 162 (87%0 

148 out of 162 (91%) 

, 140 out of 162 (86%) 



97 



expected to .achleX(e as good classif icatlfon r^ults as either indexer indexing 
the entire data base alone. Thei;^fore^; there appears to be nonnecessity to 
use only on^^ indexet^ifor a particular dkta base in order tp obtain optimum ' 
extraction of differentiating information. These conclusions are based qn the 
premise that .in.dexers are well trSined to^egln with and conscientious in 
applying the indexing »irul®9 and convetttion>s to the best of their ability. 
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SECTION 5.- EFFICIENCY OF TH£ RATIONAL CONDENSATION SHORT-CUT INDEXING 

PROCEDURE COMPARED TO THE ORIGINAL LENGTHY INDEXING PROCEDURE 

■ ■ ^ . . ■ ■ . V ^- , - 

IntxodMCtion 

Although the rational condensation short-cut indexing procedure did not 
achieve the classification accuracy of the original lengthy indexing .procedure, 
which had mote variables available for the stepwise discriminant analysis pro-* 
cess, it did achieve an acceptable level of classlf icat:^nperf ormance in cdm- 
parisotr to the longer, more complex iiidexing methodolog:^FT\^e rational con- 
densation sha»t-cut indexing procedure is much easier to apply and appears to 
be slightly more reliable since therje are fewer areas of ^mbiglPlty, resulting 
in more consistent interpretation of the indexing rules and conventions • 
Therefore,' 'an important consideration is how efficiettt each of .th^e two in-* 
dexing procedures is for indexing a particular sample of Evaluation Reports* 
This kind of copiparison can provide data needed for assessing the economie" 
feasibilrty of adding' information extracted from narrative comments into a 
comp9site score for predicting an. enliffted man's potential for assuming the 
managerial responsiljilities of the next higher pay grade. 

^ ' \ V ^ • • 

Methodplogy 
^ ■ , ^ . ■ • 
A subsample. of 12 Evaluat^^idti Reports takSrfrom the E5-E6 fleets trial 
data base was selected, ranging from a report with * short narrative text on 
only the Evaluation Section (43 words - Case 406^>,to a report with v^y 
lencthy narrative comments on both sections (378 words - Case 717),k^^4:]^ time 
required to applyxthe rational cJmdensation sht)rt-cut indexing procedure and 
the original lengthy indexing procedure to this 12-^>ase subsample was com- 
pared. Taken into account in the comparison was tlie time required by both 
indexing procedures (1) to index the narrative text ifor the Evaluation "Section 
and the Justification Section, (2) to enter the resulting indexing decisions 
onto the indexing form, (3) to code * the data recorded* on the indexing forms 
onto IBM coding sheets, (4) to keypunch the IBM coding sheets, l^nd' (5) tb. 
keyverify the IBM coding sheets. The computer pro^nesslng time Required by 
each of the two indexing procedures "also was considered. 

•' . ' ' ■ ■ •A 

t Results , . • 

I ; ^ <c ' ^ / ' ^ . • ^ 

' In comparing the efficiency of the two indexing j^cedures pn the*12-case 
subaatople described above, the .Evaluation Section and th^ Justif icat-iron S^c- 
tion were considered separately^ where possible. Table 33 shows the results % 
tor^the Evaluation Section. In the application of^lre^h indexlng^^ methods^ the 
total numbemof words contained in the narrative text must be counted, and the 
time to make this count is the same for both methods. For the Evaluation 
Section, the total number of wqfrds ranged 'from 36 to 195, with a mean of 80 • 
woifds for the 12 cases. On the average- it took 45-1/2 seconds to count the 
Words in the narrative text of the Evaluation Section, or approximately three- 
quarters of .a minute/ The average time required to index the Evaluation 
Section for these 12 cases was approximately 5-1/2 minutes for the original' 
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letij^tliy indexing procedure and approximately 3-1/2 minutes for tjhe' National ^ . 
condehsation.shprt-cut indexing procedure.. An average of 8.8 index terms were 
assigned" using the lengthy procedure compared to an average of I'O.l :^dex , 
terms- using the short-cut procedure. The reason that; more index terms were. ^ 
applied in the, shor^^-xzut procedure i^ that a greater degree af indexing ex- 

' haustivit^ is/exercised, but since there is' less room for ambiguity irin inter- 
•pr6ting the l/tidexin& rules,^and conventions for the rirtional condensation , 
short-cut Vethod, selection^of index terms c^h be accomplished more speedily, v 

-Miy there id lesa room for . ambiguity is explained on page 50. 

Ttfe average time required to^ enter the indexing decisions made on the ^ j 
^Evaluatibn Section onto' the indexing form was 2-1/2 minutes for the lengthy 
^indexinp prpcedure and 1-2/3 minutes for the short-cut indestlng procedure- 
The '^Ime required to code the data .recorded on the indexing form onto an IBM 
coding shee't J^s "where the most signifiicant time difference occurred between 
•the 'two indexing_procedures. For the Evaluation Section it reqijired approxi^ 
mately' 2-1/2 minutefe on the average' to , perform- the qoding operation for th'e / 
lengidhyo method compared to a little under I minute.for the^short-cut method. 
This result is not sutptising since two punched cards uti^zing a total of 149. 
columns ' comprise the -coding fdmnat" for, the lengthy, method whereas only 55 
colpmns contained ori onp punchjed card comprise the codirig foinnat for, the „ 
' rational condensation short-cut method/ 

^ ' Tabl« 34 presents the results for thte Justification Section. Th6 total 
*^ number of words contained iii the narrativeX tyaxt ranged from zero words (two 
cases had" no i>is.tif icatibil comments) to 143v words, with a mqan of 61.3 words 
for the 12 cases. On the- average it took afypro^imately 35 seconds to count 
\he words iii the narrafiive text of the Justifdcation Section, or a little over 
' half of a minute. Tb0 average time required eo, index the Justification. Sec- 
. tioii for* these 12 cases, was^a little over 5 minihres " f or J:he length^ pro^jedure 
and 3-1/2 miriutes for the short-cut pracedure. An. average of 6.1 index terms 
wejre assigned using the lengthy procedure compared to an average of, ^-8 Index 
terms u^iifg the short-cut procedure. " , . • ^ , 

, • , . > ' \ V ■ . 

•The aver^^ge time required to enter the inijlexing decisions made on the 
■ Justification Section onto the indexing form was 2 minutes -Jbr the lengthy _ 
, indexing probedure' and 1-1/3 minutes for the short-cut Indexing. procedure . \ 
The time required to code the data recorde'd on the indexing form onto an IBM ^0 
^coding sheet was a little over 2 minutes fori'the lengthy procedure compared to 
approximately three-quarters of a minute for the short-cut procedure. 

' t-tests of mean difference wej:e "computed "for' these; four comparisons on 
both the Evaluation Section and the Jifetification. Section^ The results aire 
. reported in Tabl^ 35. ' -Althou^hfit takes less time' to 'i»aex with^tWshort-cut 
indexing procedure, this .difference is not statistically significant on either 
•the EvaluationySedtion or the Jus^ficatio^ Section. HoWever, in interpreting 
this result it should be borne in- mind that besides -the smallness of the sub- 
. sample of Evaluation Report;s used in this study^(only 12 cases), the cases 
J^ere selected ta cover the full range of length. of ' text . This latter charac- 
teristic of the subsaraple served to increase the variance, thu^ requiring a 
Idrge^mean difference to affchieve statistical sfgiiif icabce.' , In actuality 
there is a consistent difference in the length of time needed to apply the two 
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indexing procedures, with the rational condensation short-Cut procedure taking 
approximately two-thirds of the time required by. the original lengthy proce- 
dure. Indexing time is the largest component af thte total time required 1;o ^ 
apply a particular indexing procedure. Consequently ^ if indexing time can be 
reduced, then 9ignifi?cant cost savings can be realized. ' ^ ^ ■ 

Table 35 ^^Iso shows that although more index terms 'are assigned in the 
application of the short-cut procedure compared .to the lengthy procedure, this 
difference is not statistically significant on either the Evaluation Section 
or% the Justification Section. However, the average time r^quir^d to enter the 
indexing decisions onto the -Indexing form was significantly less ^ for tVje 
short-cut procedure on both sections of the Evaluation Report. On the Evalua- 
tion Section this difference was statisticall^^ignif icant between -the .01 and 
.001 levels of probability; ort the Justif i^«€ion 'Section this difference was 
statistically significant betnSreen the .OJir^and .02 levels of probability. The 
Aiost pronounced difference statisticaily between the two Indexing' methods 
resulted in tjie t-tests of mean difference in the*average time required to 
code the data recorded on the Indexing forta onto IBM coding sheets, with the 
short-cut procedure requiring significantly less time than the lengthy proce- 
dure. On both the Evaluation Section and the Justification Section this . 
difference was statistically significant beyond the .001 level of probability. 

The analysis of the keypunching and keyverifying of the IBM coding sheets 

pertaining to the 12-case subsamplo indexed by the lengthy and short-cut 

indexing procedures was carried out? in the following manner. First, the time 

required for keypunching and keyverifyiog the Evaluation* Section and the 

Justification Section could not be ^^eparated in the analysis since the cards 

pertaining to these two sections for a Case are punched or verified sequen- ^ 

tially, not separately. Second, if the machine operator had stopped to record 

the elapsed time for each case individually, it would have interrupted her 

Ikeyboarding rhythm. It was not practical to have an observer, attempt to clock 

the elapsed time for each case since the only clues to rely, on are the release 

of a. card from the punch position and the registration of the n6xt card in the 

-punch position, a sequence of operations that would.be impossible to clock and 

record while also noting the beginning time of the next case. Therefore, the 

times required to keypunch and to keyverify the IBM coding sheets pertaihing 

.to the 12-*case subsample were recorded In toto for each indexing method. 

The^e total timed then were divided by 12 to arriye at the average keypunch^.ng 

and keyverifying time per case. . 
/ ^ • o . J 

^ For the lengthy indexing* procedure, on the average it required 61.4 
peconds or a little over a minute to keypunch the two cards -cqXresponding to 
each eas^ compared to dn avera^ of 29 seconds or approximately half a minute 
to keypunch tl)e single card for each case indexed by the -rational condensation 
short-cut indexitjg procedure. The average time required to keyverify the two 
punched* cards for a case ittdexed by ttie lengthy procedure was 53.7 seconds or 
,^ little undef a minute compared to an' average of 23 seconds to 'keyverify the 
single punched card for a case indexed by the short-cut/ procedure • 

In an effort to arrive at some estimate of tbe comparative costs of 
applying ^he. two indexing procedures to a typical* 1067ca8e sample of E5-E6 
Evaluation Reports, Table 36 "wafi^ prepared . The cost estilnates in this table 
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include^ all of the steps in the content analysis of the narrative' text up to 
the point of computer analysis. The average time required by each step for 
the Evaluation Section and Justification Section combined was used as the base 
and multiplied by 100 in order to arrive at the estimated .time ^requited for a 
sample of IM^ea^Ss. Thus, it can be seen in Table 36 that an estimate of 
17.7 hours/is needed to index the narrative comments in both sections of 100 
E5-E6 Ev|dluation Reports using the lengthy indexing proce4,ure coftpared to 11.6 
hours using thewrational condensation short-cut indexing procedure. At an 
hourly /rate of $7.00 for the indexer, the estimated indexing tpst for the 
lengthv procedure amounts to $123.90 compared to $81.20 for the ^hort-cut 
procedure. . • . ^ . , 

Whatever hourly rates might be appropriate for the local situation can be 
substituted in this table in order to arrive at overall cost^stimates, for the 
two indexing procec^ureo. The hourly ratee that were used^in arriving at the 
cost estimates are representative of^ the actual costs incurred in this, rer 
search for ^the various steps in the content analysis. Probably the most 
informative cost es^timate is the total estimated cpst for applying each of the 
two indexing procedures tJb a 100-case sample of E5-E'6 Evaluation Reports. Fo;r 
the original lengthy indexing procedure, the total estimate^ cost is $202.30 
up to the point of computer an^ysi^s, or approximately $2.00 per/caseC For 
the rational- condensation short-cut indexing procedure, the total* estimated 
cost is $119.70", oi approximately $1.20 per case.* This cost comparison sug- 
gests that tl)e rational condensation s'hcnrt-cut indexing procedure can be-, 
applied for about 60 percent of the cost of applying the lengthy indexing 
procedure tolthe same corpus of narrative text. Since little' is lost* in 
classification accuracy by using- the short-cut indexing procedure, the eco- " 
nomlc advantage of this indexing method opts in its favor. 

The only a^spect of* the comparative efficiency of thq^e two indexing 
procedures that has not b4en considered thus far is the cost incurred by each 
procedure £^ the stepwise discriminant analysis. This cost primarily^ is a 
function of .the size of the S4mple being analyzed and at how many steps, in the 
process the computation of a classification matrix is specified. The entire 
datQ set must be scrutinized each time that the computation of a. classifica- 
tion matrix is specified. The image of two punched cards per case (lengthy 
procedure) takes more time to scrutitiize than t^ie image of a single, punched 
card ^er case (short-cut procedure). Ho^/ever, the computational costs are 
insignificant compared to the laboi^- intensive steps in the content analysis 
that are included in flTablc 36. Although computer costs would be dependent on 
the billing algorithm used by the computing facility at which the- stepwise 
discriminant analysis was' performed, a rough but probably reasonably accurate 
estimate Is that the cost to perform a stepwise discriminant analysis of a ^ 
particular lOO-case sample would amount to approximately 2. to 3 percent of all 
of the labor coats incurred preparatory to computer analysis. The difference 
in cost In favor of the short-cut Indexing procedure for t;he computer a^nalysis 
is Insignificant, amounting to only two or three dollars foK^ sample size of 
around 100 cases.* - ■ 

9 

The overall conclusibn that can be drawn from this study of the compara- 
tive efficiency of the two indexing procedures is that the rational condensa- 
tion short-cut indexing procedure is moire ^ost effective than the original 
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lengthy indexing pTQ<iedure. The only justification for using the longer, Aiore 
complex indexing tnethodology might be in situations where it was expected that 
discrimination bfetween criterion groups would be extremely difficult to 
achieve, for exatnple, selection of senior chief petty officers for advancement 
from Pay Grad^ E8 "to Pay Grade •Eg (Master Chief Petty Off icer) , where the can- 
didates for promotion constitute a small, horapgeneous group of highly quali- 
fied 'chi^S. The original lengthy indexing procedure has shown itself, to 
yield slightly better classification accuracy than the rational condensation 
short-cut indexing procedure because it has more variables available for thp 
stepwise discriminant analysis process,^ in situations where discrimination 
is 'expected vto be difficult, the original lengthy Indexing .procedure shooXd be 
given serious consideration if the budget allows for its application. 'Other- 
Vise, utilization of the rational' condensation short-cut indexing procedure is 
recommended because it is substantially taore cost effective and considerably 
easier and more comfortable for the indexer to .apply. - 
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SECTION 6. FURTHER AREAS OF INVESTIGATION ^ 

/ 

One final task refliains to be done to complete this basic research project 
namely,' to perform a replication of the content analysis of the first E5-E6 
fleet trial sample. Since six occupational specialties were represented In 
the first SOQ-case E5-E.6 fleet trial sample, some of the sijbsample sizes for 
the six occupational groups are not -as large as those for the four occupa- 
tional specialties represented In the three E7 sampii&s studied earlier. Con- 
sequently, .the results of the stepwise discriminant analyses performed on the 
six occupational specialties represented In the first E5-E6 Fleet trial sample 
should be regarded as suggestive rather than conclusive. - Therefore, in t|ils 
final task the number of cases In eacH occupational specialty will be doubled 
In order to" accomplish two objectives: (1) to perform a replication of the 
analysis of the first ^OO-case E5-E6 sample on a second similar sample, and 
then (2) to combine the two E5-E6 samples Into one large 600-ca8e sample In 
order to provide more substantial subsaiQple sizes for the analysis by occup.a- 
tlonal specialty. In addition, another occupational specialty — ^^Hospital 

Corpsmaa (HM) will be added to the replication sample , making a total of 

, seven occupational specialties to be represented In the secoflicl E5-E6 fleet 
trial sample. ' , . - 

^ Table 37 suironarlzes the various subsamples that will have been ana^lyzed * 
at the conclusion of thl's basic research project. Exploltaitlon of^ the sub- 
stantive Information contained In narrative performance evaluations for 1,328 
enlisted men then* will have been carried ouj: across three pay grades for ten 
occupational specialties. Note that by dou'bllng the original E5-E(i fleet 
trial sample ,^one of the ten occupational specialties will be represented by 
less than 60 cases. The results of the various stepwise dlacrlminant analy- 
ses", therefore, should provide stable results for use by NPRDC in conducting 
subsequent applied research studies that Include performances variables derived 
by the content analytic techniques developed In this basic 'research project. 

It should be pointed, out that the replication of the original study/ "on 
the first E5-E6 fleet trial sample will provide a more stringent test of phe 
rational condensation short-cut Indexing procedure. In the original study of 
the first E5-E6 fleet trial sample,, the individuals marked In the flower por- 
tion of the marking scale on Perforroahce of Dfity Uere eliminated from the 
sampllnp, paradigm. The sample cases then were drawn^ from the remaining top 

partlon'of the marking scale sd as to form three crlte,r,lon groups Upper, ' 

Middle, and Lower. ' Within this top portion the three criterion groupji wore * 
selected to be as widely separated as pooslble, ^iven the available cases from 
which to sample. In the replication of the original study on the first E5-E6 
fleet trial sample, the three criterion groups, by necessity will have to" be . 
much more contiguous on the criterion variable. Performance of Duty., Conse- 
quently, the ability ofthe stepwise discriminant analysis algorithm to cor- 
rectly classify •individuals Into their appropriate criterion. group u«lng the 
quantitative variables derived -from a content analysis of the narrative evalu- 
ation comments using the rational condensation shopt-cut indexing procedure 
will be much more severely tested than It was in the original .E5-E6 study, 
^Nevertheless, It la anticipated that the rational condensation short-cut 
indexing method will be robust enough to perform well 'even under these niore 
challenging circumstances. * ^ , • - -* 
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PURPOSE 



The purpose of this manual is to train nonre^earchers In the content analy- 
sis technique developed Iti a peirspimel research study by l^K Research and Sys- 
tem Design to^analyze the narrative sections of Navy performance evaluations • 
for naval enlisted personnel in Pay Grades E5, E6, «?, E8, and E9.^ Pay Grades 
E5 (2nd Class Petty Of fleer) , J16' (1st Class ?eity Officer), E7 (Chief Petty 
Officer) , E8 (Senior Chief Petty Officer) , and E9 (Master Chief Petty Officer) 
ar^ all petty officers In the U.S. Navy. The objective of this study was to 
provide personnel decisions makers ((|.g., selection boards and detaflers) with 
a standardised way of detecting vallcj and discriminating Indicators of on-^job 
^performance in the narrative comments written b^y evaluators so that they might 
choosS the most qualified candidates fo* promotion, assignment, or quality- 
retention. ' ' ' • ' 



BACKGROUND • 

Evaluation marks are assigned to an Individual on a ntimbar of items by the 
eval^tor In a Performance -Evaluation Report. The purpose of this evaluation 
Is TO compare the Individual with all others of his rate known to the evaluator 
on specific aspects of on-^job performance. The evaluation marks are used iri 
making personnel decisions fbrore^enllstment, advancement, and awards'. Two. 
sectloiis of the Performance Evaluation Report provide space for the .evaluator 
^to.wrifce narrative comments to describe further the individual's performance 
and qualifications. The two sections are an evaluation comments section and a 
justification* comments section, the second of which. Is required to be f illed ^n 
for individuals whose evaluation marks were at the high end* of the marking 
scale. * ' » 

The EvaJjiiatlon Section and the Justification Section are referre4 to as 
the narrative text of the Performance Evaluation Report since they are the only 
portions of the report where the evaluator uses his own words to assess the on^ 
'job performance of the enlisted man that he is evaluating. Thus far the nar- 
rative Evaluation and Justification Sectiohd of the Performance Evaluation Re« 
port have not beer^ exploited systematically in making personnel decisions be^- 
cause narrative text tends to resist obJe(!tiye analysis and interpretation. 
However, rc^sults' from previous content analysis studies of the narrative text 
strongly suggest that there are stable differences among petty officers in 
their performance characteristics that are reflected in the narrative state- 
ments written about- them by evaluators.^ •^^^ , Furthermore, these differences 
are bbth*ldentif iable and quantifiable* The remainder of this manual presents 
a set o£ explicit and detailed guidelines for identifying, indexing or labeling, 
and quantifying (by means of a weighting spale) the concepts and ideas repre- 
sented ih the narrative text o^^Navy performai^ce evaluations for enlisted per- 
sonnel. These quantified labels have been shown to discriminate or differenti- 
ate betyeen superior enlisted personnel and their 'slightly less qualified col- 
leagues. 



112 



j ^ ■ ■ - THE MANAGEMENT PROCESS . 

Th^ operations o^a. manager may differ from one organization or from one 
institiitionai setting td another; however, the function^ of a manager are com- 
mon to all. "^The task of the manager is one of selecting goals' and designing 
and maintaining an environment that makes possible the performanpe of indi- 
viduals working together \n a group to attain these goals. An example of the 
managerial prpcess in operation 9an be illustrated by the task of finding a 
suitable home" in whidh to live. The goal selected is to find a home for X 
amount of dollars within a certain geographic area. After using the available 
resources, such as real estate agents and newspaper ads, it becomes evident 
that^the only way in which to attain the goal is to build a house. The selec- 
'tion of this alternative and how to implement it is all part of the planningr 
process. Information must be gathered, decisions must be made on the basis of 
-this Ihformation, and the means for accomplishing the goal must be decided upon. 
The budget then is specified and <^1 located. \ amount of dollars will be slpent 
on the Idt aruj Z amount on thfeJhouse construction. Other decisions that must 
be made are how large the house should be, what style it should take, what •• 
materials should be used considering the budget constraints and the style of 
the house, how long the building stage should take in order to meet the needs 
of the new occupants and still be a realistic compromise with their expecta- 
tions, how many people are needed to implement each stage of the plan, what 
their, backgrounds sliould be, and so on. Once the plan is established, it is 
necessar-y to ensure that the plan is carried out so that the goal set. 
reached. This is accomplished by the contiolling function. Performdnce of the 
tasks involved' must be measured constantly against the plan, and the, correction 
or prevention of deviations from the plan must be 'monitored continuously. 
Through the accounting systelh, it may be discovered that too much of the budget 
was . spent at one stage of building so that cuts must be made at another stage 
to bring thp budget back into line with planned expenditures. The time spent 
on grading ^he lot may^ave taken longer than planned and time-saving devices' 
may b^ employed at a later stage of building to adhere to the plan. Or the 
plan may have to be altered and a later date to move into the new house may - 
have to be established, thus causing' an alteration of related factors such as 
extending the lease for the house being rented on a temporary basis until the 
new house is ready for occupancy. Contiolling takes place throughout the 
entire building process, correcting deviations and preventing deviations from 
the plan in order to 'meet the established goal. 

The construction has to be organized, a task which involves the organiza- 
tion function. The number of activities are enumerated, such as designing, 
contracting, and subcontracting. Authority relationships- are estal)lished. The 
designer and the general contractor report directly to the owner; the sub- 
contractors (framers, carpenters, plasterers, painters, etc.) report to the 
•general contractor; and the designer and contractor confer ;with each other on 
■ an equal authority basis under the owner's supervision. All these- roles and 
their interrelationships are part of the organization. structure.- The staffing. 
function concerns manning the roles or activities that have been enumerated in 
the organization structure. Individual^ are appraised, selected, and hired. A 
designer is selected to perform the designing activity instead bf an architect 
since it was decided during the planning stage that designers are less expen- 
sive on the cost side and not much is lost on the benefit side. ,If some 
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•Indlvidu'als have to be trained on the job, this is all part of staffing y that • 
is, keeping laanxied the roles as specified on the organization chart* Once the 
pqsltlon^ ari^ ^nned, constant controlling or monitoring Is needed to ensure 
that the jobe^ are being performed as planni||l. " , o * 

.-^ . The leadejpship and directing function is performed by the;^6wner and also 
by the general contractor when the subordinates or Subcontractors are .motivated, 
guided, and;^ supervised in accomplishing a task and working towards improved 
performance.' The designer may also have a staf^that needs to be' led and * 
dltected. Subordinates' are encouraged to wo^k together harmoniously and in an 
effective and efficient manner with the aim of^ achieving the' primary goal, * * 
Through the performance of these bas£c managerial functions , the m^ttf ^al of 
having a suitable house In which to live is aqhleved. , ^ - 

For certain organizations there Is one other manageme^ function of sig- 
nificant note-— representation. 'This , function refers to the- creation of an 
image" of an organization to the external or internal envirolxt^jt^ntj ^|?he Navy is 
a service organization^ and how it is viewed by the i^ittllc ^an influence its 
funding by the Congress. How the institution looks to the outside yorld 
more important for the Navy than for a private corporation. Also, in ordefr to 
attract recruits , t;|ieir Image Is Very important*^ * , ,/ 

k*^ ENLISTElp PERSONNEL AS MANAGEflS . ' 

Enlisted .personnel in Pay Grades E5, E6,-E7, E8, and E9 are all managers 
in the sense that they all are responsible for the aupervislon of bther 
11s ted men whose work they, direct.* Therefore, the unifying focus in^h^^^^^' 



ioanual is on the assessment of a petty officer as ^ manager, v Petty '^jSJ^rs in 
these pay grades are junior level managers, and as such, -they must p^|form * 
technical as well as managerial functions. . ^ ■ / " 

Table A-1 shows a hierarchy of 15 index terms or descriptive labels that 
can. bemused to characterize the on-job managerial performanqip of petty officers. 
These index terms are the^ terms to be mapped onto the narrative text to give it 
objective structure and to systematize the way that this text is analyzed and 
Interpreted. Note In Table A-1* that the 15 index tehm are divided into three 
sectlbna* * The first section contains four terms wliich represent seven MANAGE- 
MENT FONCTIpNS that many aiithorities on management practice agree 'are the 
characteristic duties of ^I'managerg*^'^*^*® Although' some authotfities be^- 
ll,eve that there* are more, less, or diffeirent functions performed by managers, 
these seven, functions (^condensed into four terms) were selectted bejpause they 
are representative of the duties that petty officers actually perform, v' 

The second section of Table A--1 contains seven index terms ^or different 
types of SKILLS AND ABILITIES considered^ to be important by Navy* supervisory 
per^sonnel in performing effectively as a petty officer. While some authorities 
on management practice .consider making a judgment about whether or not an indi-, 
vidual possesses* a skill, quality, or ability to be a subjective process, Nayy . 
evaluators do repeatedly call out these specific qualities in their narrative 
evaluitioni becaiise many of these qualities Are dimensions on which the evalu- 
ator assigns evaluation marks to an individual in the Performance Evaluation* 



103 



Report.. The first section x>f -Table MANAGEMENT FUNCTIONS; deals with how 

a ratee .performs his manager ial*' functions: ^nd is result oriented, while the 
second s&c t ion~-SKILLS AND ^ABILITIES— contain? index terms that relate to an 
individual's charact^ristids, and qualities which, if used, may help him achieve 
good results. SubsumeV under CONDUCT AND AT^TltUDE are the more specltic types 
of conducts andl^ateitud^s COOPERATION AND RESPONSIVENESS and. ENDURANCE AND MO^ 
TIVATION^ which often are identified specifically in the Evaluation Reports/ 

.The third section of Table A-1 PRODUCTIVITY AND ACHIEVEMENT is also a 

result-oriented section of the hierarchy. Hei;e 'are included the measures of , [ 
overall performance. Subsumed under PRODUCTIVITY AND ACHIEVEMENT is RECOGNI- 
TION which represents the acknowledgment of an individual-* s 'per fotmanae. 
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QUANTIItING THE itDEX TERMS * 

It is not enough to simply label a narratlv-^" statement^with tt^mo^ ap- 
propriate index tepin since' the statement^ may have beeH a highly posTt*^, quite 
positive, neutral, quite negative, or highly negative one.<. For example, in 
order to differentiate, between the 'indivijiual: who plans supevbly and the indi- 
vidual who plans "inadequately, a weighting acal6 was deyised to be applied to 
each index term that is us^ (see Table A-2) . The weighting ''scale contain^ 
five niimerical yali^ea rangitxg from 3, the positive end of- the scale, to -2, ,the 
negative jend of the scale.* l^er each numerical value in Table A-2 there, are 
listed examples of descriptive^ords or phrases that may be u^e^by the ev^lu- 
ator to describe an individual' s 'per fotaanfce. These lis<:$ of wbrds provide 
clues to the indexer as to which numerical value' to assign to an index teirm^ • 
As a simple example, if the evaluator commented that an individual was "highly 
cooperative," this statement would be indexed as COOPERATION AND RESPONSIVENESS 
and assigned a'.weight of 2 since highly, is listed as an example under nu#eral 2 

in- Table A-2. ^ V 

• ^ i 

However, when a qualifier is not ptesent on the weighting scale, the in- 
dexes will have to exercise his own judgment. ' An Important tool to use for 
making weighting decisions is a dictionary. For example, the words consitantly 
or absolute may be used as qualifiers. Synonyms for the. word constantly are 
. always r all the time, repeatedly,, and veary often. Very and\always are listed 
on the weighting scale as 2 weights; therefore, a 2 weight would .be the logical 
choiice for the qualifier constantly. There' ate severf J. meanings for absolute 
such as perfect, complete, wtfble, purSf positive, certain, and definite^ fin 
important consideration for the indexer to keep in mind is the context in which 
the qualifier has appeared. ' In the phrase "absolute loydlty," the indexer has, 
to decide which synonym best suits absolute hy trying to understand the meaning 
of the qualifier in reJLatioi^ship to-^-^ts surrounding text. In this case,, the 



* If certain index terms are not used at 511 in indexing the Evaluation Section 
narrative or the Justification Section narrative, they are given a vdluS 5f 
zero on the weighting scale. ''Zero weight which represents no comment^ is placed 
between the positive comments (3 weights* 2 weights, and 1 weights) and. the 
negative comments (-1 weights, and -2 weights). However; the. indexer should , 
ignore the zero value for indexing purposes since it only becomes important 
when the. indexing decisions are recorded on the ^.ndexing , forms* 
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evaluator seems to be saying that the Individual "completely loyal," so com- 
plete would.be considered to be the appropriate, synonym 'for absolute. Since 
complete is a 2 weight, absolute would be assigned^ a 2 weight* - In ^ity^atiohs 
whete \yn0nym3 fall under different weights it is useful to ask, "Could this 
'statement* have been phrased ^n another way by the evaluatot that would have 
madeii, it a stronger or' a weaker statement?" The evaluator could- have said 
"maximum loyalty" or "supeS^b loy^lfcy" which thenVould have warranted a 3 
weight, ^ , .* ' ^ • . 

• *" ♦ ' ' • ■ . 

The' Comparative qualifiers can be used in either a positive or negative 
connotation- 4epending on the '^surrounding context of the text. "For example, "He 
is very trustworthy," would be indexed as CO!toUCT AND ATTITUDE 2 because very • 
falls under the 2 weights on the weighting scale. However, if the context of 
the text was a negative *one, "He is very untrus^tworthy/^hen the weight would 
fall in the same place oh the negative side of thp^scaTe and the l^ibel CONDUCT 
AND ATTITUDE -1 would be used. Note that an award or a T)Uni8hment is given 
either a 3 or a weight with the Index t%nii» RECOGNITION since ^^ere is no 
degree of variance. Either the individual was given an award' or not,^ of was 
punished (disciplined) or not. . ^ S ' ^ 

^y' . * ■ ^ ' - v. - • , 

- k \ . -SPECIAL. INDmNG .CONSIDERATIONS 

An alphabetical dictibnary of the 15 itidex terms appears At the end of 
this discussion, ipoif each term iiji thfe dictionary 3, a definition is given, ex- 
amples of narrative text Indexed with the term are cited, and usage rules to ^ 
guid(^ the indexer in choosing this term or another term are supplied. Careful 
study of the dictionary will instruct the new indexer in how index terms and 
their mnaerical weights should be assigned in order to ensure a systematic and 
objective application of the indexing procedures explained in this manual. 
Although some indexing examples may not always seem logical to the new indexer, 
each indexing decision has been meticulously and thoroughly considered. The 
examples presented in the alphabetic dictionary represent a distillation of 
three years of indexing exp/arience and constitute a self-instructional compila- 
tion of crucial index;ing rules and conventions that the new indexer needs to 
l^ow in depth in order to be abl€ to index trke narrative text of Performance 
Evaluation Reports accurately ai^d consistently, Evetf after car6fully studying 
the manual, the Indexer should ifefer constantly to the manual while in *the pro- 
'cess of indexing. Figure A-1 presents an example of the indexed narrative sec- 
tions of a Performance Evaluation Report. . ^ 

There are several indexing considerations that should be kept in mind ds 
they will assist the Indexer In maintaining consistency and will help resolve 
indexing dilemmas, e»It must be ^remembered that concepts or ideas are being in- 
dexed and not words alone. There are two basic approaches to Indexing, One 
approach is more mechanical and the indexing is done by uae of a key word &Lp- 
proach. The second approach is intellectual ari^d the indexing Is done with a 
conceptual approach. This manual Illustrates the latter conceptual indexing 
approach so that concepts are indexed and not key words » For example in the 
keyword api)roach, "He knows how to manage his men," would be indexed as 
MANAGEMENT FUNCTIQNS because of the' word "manage*" when the concept that actu-^ 
ally Is being conveyed Is that the individual is proficient at LEADERSHIP AND 
DIRECTING. Also the following phrases would be indexed incorrectly and not 



^ M05 



- ♦ 



really be the concept that" the evaluator is trying to convey if OTily the key 
word or the incorrect key word was picked up. "Abinty to plan" should be in- • 
' dexed as PLANNING-CONTROLLING anS not SKILLS AND ABILITIES. "Understands the 
'-^ . -commmications 'system's should be indexfed as PB[0FESSI0NAL AND TECHNICAL SKILLS. 

' •' ' and not COMMUNICATION".- "He" needs no snpeTVision'' should be indexed, as CONDUCT 
AND AfjriTUDE and not MANAGEMENT FUNCTIONS. fed: the word "supfervision." 

Another consideration to keep in mind Is that w!len a description of a job « 
or Job duties,-i8 included in the narrative text, this description is not Index- 
' . ' « ed sincQ it is a f actual 's tat einent describing the qualifications needed to per- 
' form a specific job or the duties of that job. Therefore, the statemene is 
(* } abotrt the job itself and not about the individual being evali^ated. Even if 

such a statemerft is modified by. adjectives or adverbs, if still- is not .indexed 
* if it lefera to how a job should be performed and not to how the individual 

actiially performs a job. As an illustration of this convention, if planning is 
. ^ mentioned .as one of the individual's duties, and it is mentioned in they context 

of a factual statement as opposed to a statement of evaluation or accomplish- 
^ liiei\t,"then no index term would be a^isigned to this statement. For example, 

VChief XX 'is required to develop procedural methods of accomplishing the divir 
* slpn workload." Even if an adjective or adverb is added to this statement--- 
" "Chief XX is required to develop effective a6d efficient procedural methods of 
accomplishing the division workload the statement is still about a specific 
jol) duty or Requirement and the modifiers refer to how the job should be-per- 
formed4? The modifiers do not refer to the individual per se and, therefore, 
this statement should not be indexed- since it is not evaluating the individual. 
However, if a statement is a qualitative statement and refers to the individu- 
al, then it beco&es a statement of evaluation and is indexed. If the evaluator 
^ said that the individual plans well on the job, then a value judgment has been^ 
rendered about what kind of a planner the individual is.. For example, "Chief 
XX has developed effectiv»?^nd efficient procedural methods of accomplishiyg 
• the division workload;" thi^ ^atement would be labeled PLANNING-CONTROLLING 1. - 
Thd statement no longer is a factual orle but has become an evaluative comment 
about the individual rtither than a statement describing a job requiremerft. 
"Chief XX has developed procedural methods of afceomplishing the division work- 
load " would be labeled PLANNING-CONTROLLINQ 1 since the evaluator thought it 
Worthwhile to me^ition this information and the statement, therefore, evaluates, 
the individual although no modifying adjective or adverb was u.sed. Always keep 
in mind that indexing decisions are made in terms of the qualifications thae^an 
■individual actually possesses thai can aid him in perforndng a job, in terms of 
how a job is performed by an individual, or in tojqna of the results aphievcd. 

r , 

Also keep in mind that iii deciding on which numerical weight to use, siodi- 
fying adjectives and/or adverbs must be associated with th^ ddca or concept bo- / 
ing indexed and not with another idea or concept in the same sentence. Each 
stateinent indexed has to be regarded as a separate entity lest confusion and 
inconsistency result. For example, consider the following statement: His re- 
soiireefulness in completing his tasks in the most efficient and thorough manner . 
is noted." ' Host' is associated with the manner in which the 'individual performs 
his tasks and, therefore, qualifies PRODUCTIVITY AND ACHIEVEMENT^^s, a 3. The 
• individual^a.ttesourcefulness is not modified but it is stated that he possesses 
that characteristic. It helped to make the 3 weight possible for PRODUCTIVITY 
• AND ACHIEVEMENT, but the first part of the statement is only indexed as SKILLS 
AND ABILITIES 1. To be given a 3 weight, the statement would have had to^ have 
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been, "His outstanding resourcefulness in completing his tasks in the most ef- 
ficient and thorough' manner is noted." - To further illustrate this rule, "His 
congeniaiity contributes significantly to *good morale among his subordinates" 
would be indexed as CONDUCT ANP ATTITUDE 1 and LEADERSHIP ANtX DIRECTING 2 since 
significantly is associ-atbd with the contribution made to his subordinates' 
morale, which is ^ leadership. function* When >t her 5 is no modifier given for an 
evaluative* statement, a 1 weight Is assigned to the index term Selected (e.g., 
"His planning efforts have le^to.^." would be labeled PLANNING-CONTROLLING 1)^ 
Also, if a modifying adjective of adverb that falls at the 1 position on the 
weighting scal^' is inclluded in the evaluative statement, the index term select- 
ed. stiJLl would be given a weight of 'l (e.g.', "His competent planning' has led 
to. would also be labeled PLANNING-CONTROLLING l). / 

However, if an adjective occurs before a s€ring of words and phrases, 
this adjeciive modifies each ^rd or phrase in the string until there is a 
clear br%^ in the sentence structure t or until the^adjective could not logi'^ 
cally and/or grammatically be associated with a particular word or phra^se. 
Only occasionally an adjective occurring before a string of words q?1 phrases 
cannot logically ^toi/or gramma^tically be associated with the string^/ For ex- 
ample, "excellent career motivated corpsman," would be indexed as ENDURANCE 
AND MOTIVATpB 1 and PROFESSIONAL AND TECHNICAL SKILLS 3, "Excellent" does not 
logically orgrammatlpally describe "career motivated," but i^ logically asso- 
ciated with yhat type of corpsman the individual is*. However, dn most cases an 
adjective occurring before a string of words and phrases does modify each wdrd 
or phrase In the string. For ax^pl^, "His outstandipAgtechblcal knowledge and 
organizational ability^ have contributed to..." would t^indexed as PROFESSIONAL 
AND TECHNICAL SKILLS 3 and ORGANIZATION AND STAFFING 3. The ^djective out- 
standing modifies bothphrases. Note ^hat organization in the above ^example is 
referred to as a skill; yet it is placed under MANAGEMENT FUNCTIONS in the hi- 
erarchy of ind^ terms. It; is often difficult to differentiate between the 
performance of a' function and the function as. an actual skill that an indiyidu- 
al may possess. For example, there is a definite ability^ to lead or skill of 
. leadership; yfet ±t also is a very important function performed by managersi. 
These subtleties in word meaning and usage are part of the expreBsive fabric of 
the English language afid continue to plague those who strive^ to achieve pret^jl- 
sion in systematizing the informatioji content of written discourse. It is for 
this reason that Indexing concept^ and not key wo3^<:}s is preferred. At some 
point arbitrary rt&les have to be Imposed. In this content analysis scheme, the 
skill in performing a specif ic futxction or the skill in overall performance 
would be, Indexed by the index term for that specific functi(^n or performance. 

Sometimes several words pr phrases describe or qualify a specific concept 
and these several qualifiers may be of different weight values. For example, 
"He performs all tasks in a superior and very reliable manner. "Superior" is 
a 3-weight word, whereas "very" is a 2-*weight word, but yet they both qualify 
the individual's* performance* The^ rule here is to choose the highest weight 
which always takes priority when a specific concept is mentioned only once and 
hence is only labeled' once. Therefore, the above sentence would be indexed as 
PRODUCTIVITY AND« ACHIEVEMENT 3. 

Every atteihpt has been made to present the Information contained in this 
OAnual in as explicit and lucid a form as possible. However, indexing remains 
.more of an art than a science. for alL of the reasons alluded to previously. 
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p : 

Xs an indexer you will encounter segri^^nts of narrative text for^ which only your 
considered judgment can help you arrivp at the final decision. It is important 
though „ that you try to keep your, judgments as consistent as possible. The p 
best way to assure consistency is to keep records of difficult .or marginal de- 
cisions and, if possiblfiLplftf the basis on which thdse decisions were made. 
Table A-3 presents a glossary of indexing decisions .that were made bjr one. ex- 
perienced indexer to handle the appearance of ambiguous or troublesome words 
and phrases in narrative text. Uise this table as an extra indexing guide. 

It is te commended that^l^e new Indexed become thoroughly f^milid^ wlth 
this training manual before ^attempting to index the narrative sections of Per*- 
forniance Evaluation Reports .V If one can^ compare one*s id.ndependent ^xal index-* 
ing decisions with those of an experienced indexer, ^his prbcedure will' serve 
^to pinpoint areas of confusion in one's understanding of the inde^ng rules 
and conventions. Frequent rereading of, and 'reference j:o, the manjfel will help 
'tfl^^araatee that t\x% rules are applied the 3ame way from day^to day<^ 

■ '-y , * • , ■ ' ■ 

' : • \ * * SUMMARY OF INDEXING ^RULES 

1. When a qualifier is not present on the weighting scale and a didtionary has 
to be used "^to find synonyms which^re present on the weighting 0cale, the 
indexer has to keep in mind the cJlttext in which the qualifier has appeared 

2. Concepts are being indexed and not key words* 

3. Indexing decisions arcf*made in terms of the qualifications that an indi- 
vidual actually possesses that can aid him in performing a job, in terms 
of how a job is performed by an individual, or in terms of the results 

^''ilrtl^eved. Factual statements such as those describing the qualifications 
needecl ^ro perfdi*m a specific job or J:h^ duties of that job are not indexed. 

4. In deciding on which numericairweight to use, modifying adjectives and/or^ 
adverbs* must be associated with the idea or concept being indexed and not 
with another idea or concept in the same sentence. . ^ . 

5. If an adjective occurs before a string of words and phrases, this adjective 
modifies each word or phrase in the string until, there is a clear break in 
the'sentence structure, or until the adjective could not logically and/or 
grammatically b(i associated with ,A particular word or phrase. 

6. When a single concept is modified by two or more qualifiers, the highest 
weight for the qualifiers takes priority and is assigned to thS^specific 
label for that concept. ' ' m 
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• • . TABLE A-1 
HIERARCHY OF INDEX TERMS 



management functions 

leadership and directing 
„=. organization-and staffing " 
planning-controlling * 
representation 

' skills and abilities 
- communication" 

^ CONDU(!t AND ATTITUDE 

COOPERATION AND RESPONSIVENESS 
ENDURANCE AND MOTIVATION 
CREATIVITY AND INITIATIVE 
INTELLECTUAL FUNCTIONING 
PROFESSIONAL AND TECHNICAL SKILLS 

PRODUCTIVITY AND ACHIEVEMENT 
•RECOGNITION 



■A 



Indexing •* 
Abbreviation 

M F 

LSD ' 

OSS : 

REP ^ 

" ' SSA 

COMM 

COND S ATT 

COOP S RESP 

* ESM 

CHEAT S INIT^ 

JNT F 

PROF S fIS . 

i 

. PSA, 
REC 



FRir 
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TABLE A-2 
WEIGHTING- SCALE 



■ V 



.comparative 



( 



above 

ri^proach 
awesome 
beyond 

reproach 
boundlesB 
excellent 
exceptional 
e:(^traordinary 
extiremely 
far flurpasaed 
finest 
flavleds 
greatest 
highest 
Ideal 

Innieasurable 
Infinite 
leave nothing 

to be desired 
less than 

1% error 
limitless 
* maximum 
niost 
never 

dut standing 
paramount 
peak , 
perfect ' • 
sterling ' 
superb 
superior 
surpassed by 

none 
top3^€bpnotch 
toj/10% 
uturomparable 
unequalled . 
unimpeachable 
unique 
uhlliitlted 
unmatched 



unquestioned" 
. ^tmost . ^ 

without:, equal 
-without*^ * " 
exception ^ 

lOOZ 



tetter* than 
most * 



above average 


model 


admirable 


much a 


.always 


no t ewo T t hv 


« amazing 


narticularlv 


better • 




broad 


^ Tjrofound 


completely 


rare 


considerAble 




consistently 


slctilflc^tit 1 V 


con t Intintifl 1 v 




deeo 


O U 1. Cl o o wU 


distinguished 




eloGuent ^ 




eminent 

^l^Hldit EA V^%4 \^ 






truly 


ennpr* *( a1 1 v 


un coiniiion Juy 


exceeds 


tin f 51 1 1* <iT" •! n c> 

UttX Oil* l-Gl. XlIK 


excels 


, UUol.Xni.jLug 


exemolarv 


finimiit)1 


wAt Licit l> * 


vaxuaoxe 


expertise »» 


vast 


extensive • 


verv 


f ihe 


WXwIc 


frGfluentlv 




fully - * 




great- 




high/highly 




Immaculate 




immensely 




Impeccable 


» 


impress ive ^ - 




inspires 




intense 




invaluable 




invariably 




laudable 




leading 




little to be 




desired 




many 




marked 




meritorious 




meticulously 
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1 

norm 



. . ^LE A-2 (e<*}Tvr 
WEIGHTING SCALE 



accurately 



typical 



Sdeq^ate^ ' 
aptly 
^sset 
capable 
cl^tly 
coinmetidable 
competent 
concisely 
correctly 
decxdedly 
definitely* 
desirable 
easily. ^ 
effective 
efficient 
emulate* 
enviable 
q,jcgg4^;€ht 
expeditioua^^^^ 
experienced 
favorable 
generally 
genuine 
good 

improved . 

innate 

instills 

keen ^ 
. teow=-»^ibw ' 

logically 

mea$urabl6' 

moderate 
neat 

obvi'Qusly 

of note 

often 
. personal 

|>rof essiotikl 

proficient 
, .promotes 
* promptly 



quickly 

rapidly 

readily 

satisfactory 

several 

skilled 

skillful 

smoothly 

solid 

successfully 

sufficiently 

thorough 

tidy M • " 

timely , 

typical 

usually 

virtually 

well 

willingly * 




fomparative 



superlative 



not as good 
as'mastf 

■:. V. .. ■ . i 

below ' 
declining 
^ quality - 
deficiency • 
degrlfeies. 
detrimental 
difficulty 
fair' 

hWidi capped 

in need of 

inability y . 

inexperience^ 

insufficient 

lack of 

la3t 

lllnited 
loss of 
lower than - 

average 
lowering^ of 
needs to imgirbve 
negatively 
neg^ective 
*poor 
. problem 
reluctant 
shortcomings 
. ■ slow 
spotty 

suffers from: 
superficial 
unabrle 
unfortunate 
unwisely 
weak 
with t^e 

exception of 



worst 



A /* bottom - 

extr^mely^ low 
lekst 

lowest ' 
major flaw : 
maximum , 
^negative) 
minimum 

J (negative) 
poorest 
worst 



EVALUATION COMMENTS 



jin the performance of his duties, XX must 



deal witl)(' civilians from 



the technician level to the management level in order, to obtain the 
information necessary to keep his project of ficer , updated on the 
A-3 program.} His ability to converse easily and to put his point 
across has made him extremely effective in this positiph. He has 
continually put forth the extra effort necessary to be one of the 
better informed persons on thf A-3 program. Due in part to his 
efforts the A-3 program is now running closer to ^ott--schedule than - 
it h^s for a long tiiae. 



JUSTIFICATION COMMENTS ^ 

XX shows Superior traits of f lexibility |in that his assigned duties 
have nothing to do with his rating as an ADR^ {Nevertheless, he has 
assumed the responsibilities of these dutiesj and performed them in 
an outstanding manner. £He has yorked for different project officers 
and with numerous different civilians^ XX has done this to a large 
extent on his own, and by digging into various problems that have 
come up, he has been a most valuable assistant to his project officer. 



LEGEND: 



COMM - COMMUNICATION V 

COND •& ATT^ = CONDUCT AND ATTITUDE 

Em « ENDURANCE AND MOTIVATION 

P&A -PRODUCTIVITY AND ACHIEVEMENT 

P-C « PUNNINQ-CONTROLLING 

PROF & TS « ^PROFESSIONAL AND TECHNICAL SKILLS 

REC . * « RECpGNITION ' 

[ ] Menotes factual text not subject to indexing 



Figure A-1. Example of an Evaluation Report Indexed by 
the Short-Cut Rational Condensation Method. 
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TABLE A-3 

GLOSSARY OF WORD CLUES TO THE USE OF INDEX TERMS 

/ 



Acts- with ease > CONDUCT AND ATTITUDE 1 
Adaptable ^ CONDUCT AND ATTITUDE 1 
Adjustable « .CpNDUCT AND> ATTITUDE 1 
Administrator = ^MANAGIMENT FUNCTIONS 1 
Admiration =^ RECOGNITION 1^ 
^Affable = CONDUCT AND ATTITUDE 1 ^ 
Aggressive = ENDURANCE AND MOTIVATION 1 
Agreeable - CONDUCT AND ATTITUDE 1 ^ , 

Alert (except wH«en used to qualify another term) - INTELLECTUAL 

. FUNCTIONING 1 

Amiable = CONDUCT AND ATTITUDE 1 

Appraisal of personnel' = ORGANIZATION AND STAFFING 1 

Assessment of. personnel = ORGANIZATION AND STAFFING 1 ^ 

Asset - If the ratee is considered to be tlfe asset to the service^ 
then use RECOGNITION. However, if one of his traits or skills is 
an asset to the Navy or in performing a task, then use "asset" as - 
a qualifier for the trait or skill. ' . 

Assiduous = ENDURANCE AND MOTIVATION 1 

Astute = INTELLECTUAL FUNCTIONING 1 ' 

Attention to detail ^ SKILLS AND ABILITIES 1 

Attentive = CONDUCT AND ATTITUDE '1 
' Attentive to duty ENDURANCE AND MOTIVATION 1 

Background in rate ^ PR0FESSI(3NAL AND TECHNICAL SKILLS 1 ^ , 

Bearing = CONDUCt\ot) ATTITUDE 1 . ' 

Can-do attitude ^ ENDURANCE AND MOTIVATION 1 

Common sense = INTELLECTUAL FUNCTIONING 1 

Congenial ^ CONDUCT AND ATTITUDE 1 

Constant* = CONDUCT AND ATTITUDE 1 ^ 
^Coordinate = ORGANIZATION AND STAFFING 1 . X 
Decision making-^ Pl-ANNING^CONTROLLING 1 
Decisive ^ CONDUCT AND ATTITUDE 1 - 
Delegate = ORGANIZATION AND STAFFING 1. 
Deliberate = CONDUCT AND ATTITUDE 1 
Demeanor = CONDUCT AND ATTITUDE 1 
Deportment = CONDUCT AND ATTITUDE 1 
Devoted = ENDURANCE AND MOTIVATION 1 
Diligent ^ ENDURANCE AND MOTIVATION 1 ; 

Directing r in reference to overall managerial functions or tasks - 
MANAGEMENT FUNCTIONS 1 
« in r.feference to directing men = LEADERSHIP AND DIRECTING 1 
Disposition CONDUCT AND ATTITUDE 1 
Dynamic ^ ENDURANCE AND MOTIVATION* 1 . 
Eager ^ ENDURANCE AND MOTIVATION 1, . 
Earnest ^ CONDUCT AND ATTITUDE 1 . ^ - ° 

Endeavor = ENDURANCE AND MOTIVATION 1 
Example or sets an example ^ CONDUCT AND. ATTITUDE 1 

EXCEPT: leads iby example = LEADERSHIP AND DIRECTING 1 , 

(Continued) 




TABLE 1-3 (CONT*) 



GLOSSARY OF WORD CLUES TO THE USE OF INDEX TEBMS 




Expertise - PROFESSIONAL AND 'EEGHNICAL S. 
Flexible « CONDUCT AND ATTITUDE 1 vr. 
Forceful = .ENDURANCE AND MOTIVATION I \| ' 

Fotcefulness of expression « C0MMUNICAT10^|;1, , , %° 

Foresight - =^ CONDUCT AND ATTITUDE"vl' • .|%> ' . ' f 

Friendly - CONDUCT AND ATTITUDE 1 V '^'r 
Genial - CONDUCI?' AND ATTITUDE 1 . 
Helpful - CONDUCT AND ATTITUDE 1 /- 
Humor and good humored = CONDUCT AND ATTITUDE 1 
Imagination » CREATIVITY AND INITIATIVE 1 
Ingenuity « CREATIVITY AND INITIATIVE 1 ' ' 
Innovative - CREATIVITY AND INITIATIVE 1 
Insight - INTELLECTUAL FUNCTIONING 1 
Insti,tUted « CREATIVITY AND INITIATIVE 1 
Inventive * CREATIVITY AND INITIATIVE 1 
Inventory - l^tANNING-CONTROLLING 1 

Job Titles are not indexed per se. However, if they qualify the adjective 
preceding thein^ they should be considered as giving additional Informa- 
tion. This happens when the text describes a skill or knowledge in a 
specific role, such as Radioman or Instructbr. (CEhief Petty Officer is 
not a specific role.) If we have a label for the specific role, then 
we would label it accordingly. For example, leader »< LEADERSHIP AND 
DIRECTING. Instructor includes more than the skill of communicating. 
It also includes skill with students, disciplining, and organizing 
material. Therefore: ' 

ProJ^essional Chief Petty Officer (CPO) - PROFESSIONAL AND TECHNICAL 
. • SKltLS 1 

Proficient CPO « SKILLS AND ABILITIES 1 
, Skilled CPO » SKILLS AND ABILITIES 1 . 
Knowledgeable CPO PROFESSIONAL AND . TECHNICAL SKILLS 1 
Outstanding man « SKILLS AND ABILITIES l' 
Forceful Instructor " ENDURANCE AND MOTIVATION 1 
Knowledgeable Instructor « PROFESSIONAL AND. TECHNICAL SKILLS 1 
Skilled Instructor -PROFESSIONAL AND TECHNICAL SKILLS 1 . 
Skilled Radioman - PROFESSIONAL AND TECHNICAL SKILLS 1 
Knowledgeable Radioman - PROFESSlpNAL AND TECHNICAL SKILLS 1 
Outstanding 'Personnelman » PROFESSIONAL AND TECHNICAL SKILLS 1 
Knowledgeable manager « MANAGEMENT FUNCTIONS 1 
^ Skilled manager » MANAGEMENT FUNCTIONS 1 

Aggtessive supervisor - ENDURANCE AND MOTIVATION I 

MANAGEMENT FUNCTIONS 1 
Professional administrator » MANAGEMENT FUNCTIONS^ 
Forceful leader - ENDURANCE AND MOTIVATION 1 
, LEADERSHIP AND DSt^CTING 1' 

Knowledgeable leader » LEADERSHIP AND DIRECTING 1 
Judgment - PLANNING-C0NTR0LLIN6 1 
Keen talttd « INTELLECTUAL FUNCTIONING 1 

(Continued) 
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. •; ./TABLE A-3 (^ONI.) 

GLOSSMt" OF %RD CLUES TO m USE OF INPEX TERMS 

Logical mind * INTELLECTUAL FUNCTIONING 1 

Loyal to the connnand (service. Navy) * ENDURANCE AND HOTIVATION 1 
Loyalty ■= conduct' AND ATTITUDE 1 * . 

Making suggestions - CREATIVITY AND INITIATIVE!. : I 

Methodical - SKILLS AND ABILITIES 1 * ' V 

Neat - CONDUtiT ' AND ATTITUDE 1 

New (something liew developed by ratee) = CREATIVITY AND INITIATIVE 1 • 
Open mindeci « CONDUCT AND ATTITUDE i - v ' 
originating ideas « CREATIVITY AND INITUTlVE 1 ' _ . 

Overcomes obstacles ENDURANCE AND M0TIVA;|fI0N 1 . ] 

Perf ectionist «- CONDUCT AND ATTITUDE 1 

Persevetance = ENDURANCE AND MOTIVATlqU 1 , ■ . ' 

Potential (referring to a general capability) - SKILLS AND ABILITIES 1 

Problem solving- * PLANNING-CONTROLLINO 1 

Quiet « CONDUCT AND ATTITUDE 1 

Recommendation for advancement ■ RECOGNITION 1 

Recommended changes » CREATIVITY AND INITUTIVE 1 

Resourcefulness « SKILLS AND ABILITIES 1 

Respect » RECOGNITION 1 

Sincere « CONDUCT AND ATTITUDE 1 ^ 
Stable » CONDUCT AND ATTITUDE 1 . ' 

Suggestion, making - CREATIVITY AND INITIATIVE 1 . . . 

Supervising in reference to overall managerial functions (St ta^ks ^- 
MANA6EMENT FUNCTIONS 1 " 
- in reference to the supervising of men alone * LEADERSHIP 
AND DIRECTING 1 V 
Supervising men « LEADERSHIF AND DIRECTING 1 , 
Supervising tasks * MANAGEMENT FUNCTIONS 1 
Supervisor « MANAGEMENT FUNCTIONS 1 . 
Tactful (used alone) « CONDUCT AND ATTITUDE 1 ' 
Tactful with his men or subordinates » LEADERSHIP AND DIRECTING 1 
Talent SKILLS AND ABILITIES 1 • . 

Tisy * coiIduct' and attitude 1 " ' ^ 

Traffic flow pattern - PLANNING-CONTROLLING 1 ^ , 

Trouble shooting (not of a technical nature) - PLANNING-CONTROLLING 1 v 
Understands (a role or policy) - PROFESSIONAL AND TECHNICAL SKILLS 1 , \ 
Versatile - SKILLS AND ABILITIES 1 , ^' 

Vigor - ENDURANCE AND MOTIVATION- 1 

Well liked - RECOGNITION 1 ' ' , 

Zeal » ENDURANCE AND MOTIVATION 1 - 
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ALPHABETICAL DICTIONARY OF INDEX TESMS 



Format: The name of each index term is"sliQvn . 
in all capital letters at the beginning of the 
definition of the term. The last section of the 
format presents a ntunber of examples of how each 
term in the dictionary should be uqed. The number 
following each itidex-term is the weight assigned 
to it by the indexer. Additional indexing rul^s 
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COMMUNICATION refers to the expression of thoughts aad^feelings through the 
spoken or written word, and the quality otf its use in the exchange of 
informatibn within an organization. Includes public speakinjg, written 
documents, and consultations. \, / , ^ 

EXAMPLES: . ' ' 

* ^ • His lise of the English language is excellent and he expresses himself 
^ exceptionally wc^ll, both orally and in writing • 

m Ratee's ability to correctly speak the English language is outstanding 
m Excellent knowledge of Engllish language 
*♦ Command of language superb 

■ ■ # 

• Grammar excellent ' 

• Vocabulary is exceptional 

m He utilizes a well-rounded vocabulary to very effectively express 
himself. 

• Eloquent speaker / . 

• Expresses very well orally 

• He possesses an average command of the English language both orally 
and in writing. 

• He is capable of expressing himself clearly and adequately. 

• His reports are accurate 

• Expresses well in writing 

• Expresses well orally 

• Speaks with eaae 

• Cj^n converge easily ^ 
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.COMHONICATION '(COKT.) 

• Relaxed group speaker 

" CCMM I ^ ^ 

• Is at ease when speaking ^ 

• Presents matter In a comprehensive and Interesting inanner . 

• Speaks correctly 
.# Speaks logically 

• Verbal expression is coinprehensive 

• Capable In expressing himself 

• Uses communication to arous6 interest ^ convince, and produce desired 
; results. 

• He is soft spoken but speaks wel^/; however, his written work, par- 
^ . tlpularly spelling, could use improvement. 

• ^Difficulty speaking to large groups , / 

COmUhlKAnOU aUo U oied wtfien kn imUvidual comunlaUu \MXh kU ■ 
6upQAXo/u ion. th(L puApo6Q. OjJ txdhanglng Xniomatlon. ^ 

• He is an excellent administrator and skillful supervisor, and he 
keeps himself arid his superloiJs fully informed of all facets of his 

^ branch's operation. ' . , ^ / 

C0M0K % . 

• He alwayb keeps his superiors informed of any problems and the status 
of work in progress. 

C0MM H C 0 

• Ratee always consults with his Division Officer concerning his wishes 
on a matter, unless time is essential. 

• He keeps his seniors alerted to pending problems. a 

a$m^ ( 

• Keeps superior well informed* 
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CONDUCT AND ATlITUDE refer to the way ^hat one .acts ^nd 'behaves towards others > 
to one*s self concept, and toy^the mental activ^-tles and attitudes^ that 
influence behavior, Includ^d^ moral principles, ^sincerity, .loyalty, con- « 
fidence, self-image, sernse of humor, courage^ flexibility, adjustability, 
appearance, grooming of one^s self and one^s clothirig, maturity, 3tabill- 
ty> responsibility, Ifevelh'eadednfess , reliability, and (Jependability. . 

■^"\ ■ ' ' ' • ■ . • " \ : 

: EXAMPLES: / V . V ' - * 



# 



His conduct and. personal appearance are always ^uperD, 

Ratee reflects^prid^^^ his pdsitj^on as a Chief Petty Officer and pro- 
vides an outstahding^^anple^through Ws ^ce^ personal 
habits and dress under all cii^cujfastances. - 

c^^F r '^TT 3 

4 He provides an^ excellent example for, his men, ^ 



Chief's conduct is never questioned. ♦ 

• He maintains an exemplary militiary appearance w>,and has hign moral 
Standards and a sound sense of values. / , 

C0t*P ^ ^tT 51 

• Exemjplary behavior \ . 

CS99P w^ATT^ C0*^9 ^ J. J x^j 

• He is always correct, and proper in al| relationships. 

• His even disposition has proven An asset' in maintaining an open channel 
of communication between instructor md student. 

C0^P r 4rT I ' • 

• Quick humor ^ , 

• Good senie of humor ' ^ • 

* * ' " -> . 

0m^P It I 
t Good tXMiple for his contemporaries 

m40P r i^'^'f r 

• GentliwMily ' ^ • ^ 

• Etijoya Ms work » 

• At e<i8e with superiors and subordinates , 

Mf^P T /ITT I 

m Not afraid to offtr celticism 

131 
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<1C^UCT AND ATTITUDE (CONT.y 



• Bordered on ln6u5ordlnatlon . * • 

^ The ratee has the potential to become an outstanding chief hut has 

problems controlling his drinking while on the bench, resulting in 
c^Xjr r ^TT-/ ^ c#>^jp ATT 

tardiness at the expiration of liberty and his absence during working 

hours. 

• Excesses in alcohol have led to tardiness and a question of hia depend- 
ability. . 

■ ' ■ / 

It mpoKtoLYvt to KmmbQjf, that m mcuttoji hovo po&itlvz a tnalt^ay bt, aX 
l6 mpoiilblz to be can^^ien^ r£n vooAj^htlnQ th(L6z tnxilt^} thoJitf^on^d, tiiz^ 
WiUQht l6 doteAmimd by thz adjzttLvz u)kitk ^uAthoA quaJU^izi thz typt 
tnaXt that an IndlvldJUJxZ po6^U6u. Fo^ •example, ohzoJiiuZ oK (ihtVif^vJL 
peA/iOmtUy M)ould bz gtvzn^ wzight 1, (jolvUt voAy chz^Aitit ok a vn/iy 
dhzQAivJi poMoncLtity maid be g^ven a^tJOOA^ht 2. Tfie' ioiZouiing pe/won- 
aJUty tAjouUtA and attiXudti muM aJU be indexed (U COUVUCX AWP A7TITUPE U\ j 
opt4ini6tLa cUtitadz}^[)l2a/>ant attUudzi taku pnJidz In kimizt^; pnJjiz tfl 
*kU> vX)Kk} pUdz tn hAA pojiiomanjuz; digntiied; 6QJil-<ioni^ld2.nt^ apHJight; 
.honut} lAJidQAt) dou not pti^c/uoubtlnatzy 6pznd6 not oxaUAlvz tiirn ^^vtAAXr 
tng"; taatiat; poA^zctlont^zi^qiUzt! anseZ^tik; cou/uigz; cou/tagt ajj 

hli cpnv^ettowd; composed; caJbn; aoujutuy; even dLUpoiJUtion) obzdlzndZ} 
toyaJUy to ki4> 6ap2AA.oU} ^Klzndly; ag^zeablz; congzniafL} gznjJxZ} - amlablz, 

Thz my that a pzA^on gKQfimi kimeZ^ and aviz& ^oK kii attOiz /izilzctA an^^a ' 
attltudz tomKu himzli and tomndA othe/u. 

• • * . * 

• Neat and polished appearance is in keeping with the highest Navy 
standards and serves as a criteria of excellence among the men wlth^ 

whom he comes in contact. ^ * . . ^ - 

* 

^•^^ f ATT 3 

• His Impecqable appearance leaves nothing to be desired » 

C0S^P Y AfT H 

• Wearing of uniform excellent 

• Ratee* 8 personal appearance is always correct and proper. O 

c0fJ9ir Art a att a * 

• He is always neat in appearance and his conduct is exemplary. 

C0009 r- ATT ^ 

• His appearance Is immaculate at all times* 

C0iii^ ^ Art X C0NP tAtt a 

• His appearance and dress is always correct, smart, and Impressive. 
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CONDUCT and' ATTITUDE (CONT.) 

# His uniform is consistently lnnna,cuXate. 

c^Mv r if rr * 

# Uniform (or dress) immaculate 

t Grooming impecc^ible 



■J- 



• His dress is impressive, and is worn with calW^, 

• He is tidy, iiltelligent, and obtains the best results from his men. 

• He takes pride in his appearance ^ ^ . ^ 

\ C09f 9 Atr I V * 

• Attention toms appearance - . X 

• Appearance military - 

«. * - ' 

m Shined sho.es 

% His tendency towards being overweight greatly detracts from his overall 
appearance. 

Statm(Ln;t& an lyidlvldrnV^ adaptability and ^^taUbUUy ^tiUct a pe/t-. 
6onalUy built o/i attitudz^ tha£ aUzcts bzhavlo/i. HovitLvoX, li adaptabiU- 
ty 0^ ^taxlbUAJty oaz quaJUileM lo^ othoJi conctptA, do mtlYidux thm. 
Fo/i QxampU, "He U a vaAy llojubU o/tgawizcA" mold bt Inddxzd 06 0R6ANI- 
ZAlnON ANP STAFFING 2. 6^ 

• His keen mind is alert to all possible circumstcTnces, and he^ succeeds 
brilliantly in adjusting to new environments. 

m Overall, he is a highly adaptable individual who exhibits unlimited po- 
tential and continuing high value to the U.S« Navy. 

• He is a mature, stable Chief Petty Officer, intelligent, adaptable, 
and reliable. 

• Adjusts quickly 

• Flexible 

C0M9 TBTT I 

m Open minded 
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CONDUCT AND ATTITUDE (CONT.) 

# Open tnlnd to criticism 

Cottcep;t4. .oi( an MdZvA^daal bting ^2JUablz, dzpmdabtz, and /lUpoM^blz cjiz ^ 
Indexzd 06 COHVUlT AWP ATTITUPE. HoweueA/ ^c^J K^Liablz oK dzpzndabtz U 
a quaJUiieA ^o/l anotkeA conczpt, do not Indzx it. Von, example, "H^ pe/t- 
^0Jtmn(iz l6 KiUUjahW' voouZd be Indexed oA VROVUCTIVXTV AND ACHIEI/EMEWT U 
and "He ^ a neJUjablz tzchn^cM' muZd'bk indexed as PROVESSJONAL kUV 
TECHNKAL SKILLS 1. 

# He Is completely reliable and has nevfer showed case^by action or ^rd ( 
to the^contrary. . , ^ . • ^ 

e00^w r ATT 3 

^ He never needs supervision 

\ % 

# He Is conscientious and Is always extremely dependable* . 

cmtt9 y hTT % ♦•iH * 

# Fully realizes his responsibilities and. at all times consciously acts 

to fulfill them ^ ' • 

C0II9 y AtT X ' I 

# Ratee Is very relj^able. 

# Completely rellable- 

# Always ready to act on his own 

# Always ready to accept additional responsibility 

# Ratee is punctual and can be depended apon to perform well regardless 

C^mp r A TT / * , 

of the amount of supervision. ^ 

A C09n^ y ATT ^ ft,A 9 €0999 ir ATT I 

He can be depended upon to "get the job done" with a minimum of supers- 
vision • 0 ^ \ 

0 ^0t4i t" ATT # 

# Willing to assume or accept added responsibility 

€0/J9 ^ ATT i 

# Does not hesitate to accept work 

^ . ■ ' 

Pt^$ ^mt^PTATTi * - 

# Works well on his own ' 

CSUP It ATT i 

# He can be counted upon. ' 
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CONDUCT AND AttlTVD^ (CONT.) 

fr-^ i . CMP r /fT-r / 

• Job done without supervision 

m He needs no prodding or prompting, 

• Can work without direction. 

^ ^ ATT I 

• Required no ^supervisiojv 

• Dependable ' ^ • c; 

• Accepted responsibilities attd authority 

C0iJy ^ ATT" I 

» Eager acceptance of responsibility / ^ 

• Assumes added responsibilities 

• Ratee's- inability to satisfactorily discharge bis financial obligations 

CmffM ATT -/ 

shows a lack of responsibility. , ' ; 

• Reluctance to sts^ume new tasks ^ * 

C0M9 t ATT ^1 T'Afr^t . 

• Need for direction and checkup by superiors 

• Occasionally needs a reminder of particularly impc|rtanp jobs. 

C««r9 V ATT -f ' 

• Relinquishes responsibility - 



"zxQmpX.an.it oetxaV/tOA." o/t ' oenav-co/i an v.xuinfj^<i. ^.um,- .,,^w n/wa 
men" iwa£d fee ind&xsd <i4 CONPUCT AWP ATTITUPE 2 and COHVUCT ANP ATTITUPE J, 
*e4pecttve£{/. However, ii a fic^eAznce, Is made to an Individual tmuna by 
■ axamplz/ tkz "example" quaUilaA the. imieMlUp abUUy oK.thz ImdeMEtp 
noJ^. Van. oxamplz, "He ImxU by zxampit," wouZd bz Indzxzd a6 LEADERSHIP 
ANV VmCTtNG 1. 
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V COOPERATION AND RESPONSIVENESS refer to«,^the specific conducts and attitudes 
that ireflect the joint effort of actinW'or working with others and the 

/ quality of readily reacting to suggestibii-^ instruction, or orders. 

i ■ 

EXAMPLESr 

• Ratee exemplifies the perfect officer-chief relationship. 
m Never hesitates to fully cooperate 

• Aatee is .extremely cooperative iii all his undertakings when «iven any 
task, and completes it in>the most expeditious manner* 

• Hfc always cooperates fully with his seniors and accpmplishes his duties 
in an outstanding manner. » * • - 

• Ratee dsAyery cox>peratlve with his seniors and his contemporaries, 
always putting the interests of the Navy first. 

• He adds greatly to the morale of the division. 

• * 
'Cmm^ ^Am9^ * ' 

• Contjinuous cooperation in all aspects . ^ 

• Co^inplete cooperation 

• He is a pleasure to work with. - * 

C00^ V- A€S^ i 

• He Will compromise* ^ £? 

€0mA r A^^^ I \ ^ 

• He will neverthelfei^s listen to the ideas, beliefs, and suggestions of 

others. , ' % 

• Gets along well with others 

• V . ' ■ ^ 

€.00^ ytA^^^ I 

• Assists others 

• He is quick to respond to any situation or problem aiid to find a solu- 
tioti. 

• He uncomplainingly responds when called upon to meet unscheduled com- 
mltments, frequently under adverse conditions* 



ERIC 



136 



125 : 

COOPERATION AND RESPONSIVENESS- (CONT.) ^ 

*• fte is .responsive and agreeable to demands upon him and constantly seeks 

L^J> / It i ^ 

ways of improving working conditions and morale* " / 

' ^ IT^ Uiif^ooperative 

• ' \ ■ * ■ . ' . . * ^ . ■ 

mold hz Indoxzd oJ^ L^APERSHIP kW DIRECTING, Homvz/t, wfien a '^tcUmznt 
hcU to do voJith an IndvjldujoiV ^ "aoopQ/icutivz^*^ (tttUudz on. "mn^king M;e££" 
mXh othoM, wfeetfie/L mpoJiLonj^ on, 6abo\dimtQ^ , thzh it ^houZd be labttzd - 
COOVEkATION m RESPONSJVEUESS. 

I • • . ■ K- ' 

• Ha is al\mys willing to help others in any way he can. 

> ■ ^ 

• Works well with superiors and subordinates 
^ Cooperative with superiors and subordinates 

• Willing to help superiors and subordinates * . 

• Willing to hel^ his subordinates 

Conazpt^ about an Indiv^Aual^^ n.2MpoMz to authonUty, command^^, n^uZu, and 
KiiQidjaUoM oAzt^doxzd a& COOPERATION AND RESP0NSII/ENE5S. % 

• He accepts authority io strfde. 

• Adheres^ to established rules and regulations 
t • Obeys all commands and regulations 



Accepts authority without question 



> 
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CREATIVITY AND INITIATIVE refer to the ability "to create, initiate, or orlgi 
nate Ideasf or tasks; characterized by being innov.aitive, inventive, or 
^imaginative. This concept is always indexed even when it is qualifying 
another Concept, 

EXAMPLES: > 

• j His personal appearance, cooperative attitude, and initiative leave 
virtually nothing to be desired. 

• He is extremely reliable and* never fails to take the initiative in 



difficult situations. 



• He displays an outstanding example in ingenuity. 

• ,Ratee displays considerable initiative in accomplishing each task 
assigned. 

• Provided Division officer with many ideas 

. ' * ^ 

• Ratee possesses a keen mind, is capable of original thinking, and ex 
presses his thoughts well and decisively when communicating with 
others* 

• Ratee is proficient in anticipating situations in his area of re- 
sponsibillty dnd initiates action to cope with them. 

• His imagination allows him to find new and different solutions to 
problems which others do not seem to be able to solve; 

• Volunteering his own views ' ' . 

CASAT r tAXT I 

• H Making suggestions (recommendations) 

^A0AT y XAfxr I 

• An original thinker 

^ ^ • .. • 

cA^^A-r r x*/tT / 

• Inventive 

<A$AT rVxT 

• Personal initiative • > 
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CREATiyiTY AND INITIATIVE (CONT.). 



• Ratee undertakes his duties willingly but lacks the initiative that 
is required of the very best in his rate^ 



However, at"^ times he lacks the dhitiative and drlVe that is necessary 
to produce these results i 
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ENDURANCE AND MOTIVATION refer to the specific conducts and attitudes that re- 
flect the inner desire and drive for self-lmprovment and to achieve or 
complete tasks, as well as the. ability to function under conditions of 
fatigue, distress, stress, and/or pain. Includes motivation towards the 
military, forceful, conscientious, energetic, enthusiasm, dedication, 
devotion, aggressive, diligent, dynamic, ambitious, zeal, - endeavor, can-do 
attitude or spirit, volunteering for work, taking courses* ' 

EXAMPLES: 

3 SfM 3 

• He is an exceptionally dedicated and hard working Chief, readily ac- 
ceptlng and expeditiously solving problems. 

• He continually strives for perfection. 

• He is extremely dedicated to his division, department, and ship. 

• Tlxe ratee always has the best interests of the Navy in^mind. 

3 

• Dedicat&d to perfection 

• His endurance is o:uts tending. 

• Always driving to do the best job possible. 

emM 9 r ^tt" / / ^ if / 

• His ability to maintain an ikij^er calm and to function efficiently dur- 

ing periods of great confusion and stress suit him ideally to his 
present assignment. 

• This highly energetic and enthusiastic young petty officer has perform- 
ed all aspects of his duties in an outstanding fashion. 

Completely dedicated Chief Petty Officer 

• Attempts to achieve perfection 

• Strives for perfection 

• Ratee performs with vigor. ^ 

• He is conscientious and is always extremely dependable. 
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ENDUKANCE ANI) MOTIVATION (CONT.) 

j 

m During the previous deployment he demonst-rated his ability to fui^ictiion 
smoothly and effectively under adverse and demanding conditions for ex- 
tended periods. ^. 

• Positive attitude towards Navy 

• He is first one^in the office each morning and usually has worked for 
an hour each morning before the commencement of ship s work. 

J 

• He takes pride in his uniform. 

• Pride in his unit or in the Navy - 

• Loyal to, the Command 

• He uncomplainingly responds when called upon to meet unscheduled com- 

^ 1^ M I 

mitments, frequently under adverse conditions. 

• Performs well under stress 

e M I 

• Persistence In the face of adversity 

6 t'M I ^^^^ ^ 

• He ,is a dedicated career man who displays pride in the Navy and the 
Squadron. o 

£ ^ M I 

• Dedication (devotion) to duty 

L r- P 9 

• Although the tasks assigned to the personnel under hi6 supervision are 
always completed efficiently and in an excellent to outstanding manner 
his superiors feel that he is capable. of getting more out of his men 
and of putting more of himself into the job. 

etM -I ^ ^ 

• - Ratee 'is not as aggressive as he could be which subsequently detracts 

Sr A ^1 

from him reaching his full potential. ^ 

' Th(L ioJUijouolYiQ iJOOKdU and phAMU mixld InddXdd. 06 EN1?URANCE AWP MOTIl/A- 
TWU 1: mf^lagglnd dUo^^ dadldotoM oK davotu long houAJ>) tUolu^} 
taking cou/UdS In oU-dvJbj hoiva} dUomlntd; mnii^ h<uid} apptidd hAmoJii 
'^^tnJjodJ^ ioK mpn.ovmwt; mnklng long kouM; agg^u^lva pMMiit o{^ diLttu; 
can-do 6plnU oKottUudo.} dAJUgant; dynamic} mbLtiouM zmly md(UivoK} 
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ESDURANCB AND MOTIVATION (CONT») ^ 

' ' . » 

tdjnacAMj} wUUng to mnk long k(nm-f viHUng WMfeet; mafeei an zHoKt. 

The. ioUming mndb and phAo&zA would fae indoxzd at, mVUmiCE AMD MOTIl/A- 
TION Zr wio/tfeA veA.y hand a^ kU Job} 6pmdi> many ext/ta hotm at hL& job} 
dzv Ota, many oU-£ittf hoimj almyi viiltinQ to mnk long hoMM wfieweye/l 
neca44Mt/; tontiniwvL6ly 6t/Uvu} tnkAjnQ many coun&zs In o{^-duty houM} 
■ d&votu many long houM. 

Whzn a ^iatmwt about d/Uvz Z6 pa/vt oi how an ln(Uvldaat U poKioHmlnQ a - 
ipzclild iuncXion oK 6kltl, Jtkm fht ^tatment voould fae M^xoJd wUh both 
conczpt6. 

• Desire for excellence In his own and subordinates' work 1L 

' ♦ Ratee works very hard at motivatitig his men to improve their perform- 
ance. - * V 

• Rate^e is constantly seeking to improve the productivity of the work^ 
center. 

• Very dedicated manager ' ^ 

€r^m% • 

• He constantly strives to Improve his overall proficiency. 

• Strives to do his very best at each job undertaken. 

• Aggressive leader * 
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ipEIiECTUAL FUNCTIONING is the ability to Xeam or understand from experience, 
and the abilitv^to analyze, reasoft,„and perceive relationships and dif-^ 
ferences. A measurement of . intellectual functioning would be a scholastic 
record. , 

- • ■ ; 

EXAMPLES: / 

4* He completed all courses with very high grades, usually leading his 
class. 

• Ratee is extremely keen minted » 

• He learns quickly and applies his training and experience effectively. 

• He is a forceful and intelligent Career Petty Officer who has such com- 
mand of the basic prof essional^tecSniques that he can ditect his ac- 
tions to job perfection and the well-being of his subordinates. 

He is tidy, intelligent, and obtains the Jiest results from his men. 
m Coherent mental organization , 

• /Coherence of his 'thoughts 
m Analytical mind 

• ^Alert' (when not used as a qualifier for another term) 

• Common sense f 

• Insight * ^ * 
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LEADERSHIP AND DIRECTING refer to the motivating^ guiding, oand supervising of 
subordinates to accomplish a job and worH towards improved performance,. 
Includes encouraging subordinates iri cooperative endeavors and ftlso in 
* self-development through counseling. Includes «the possession of! leader- 
ship qualities that can influence and affect others, 

EXAMPLES: 

• His excellent leadership qualities were especially manifest; fwhen he 
was assigned to lead the maintenance efforts of CQ.. detachments in GON- 
STELLJvTION and INDEPENDENCE. J 

• GetJir^he most out of his men *" 

i. » 3 ' 

% Best results from his men 

• Chief has demonstrated the potential xo be an outstanding leader* 

• He spenlt a great deal of extra time with hla men and turned Out well 
tralnedy very motivated men for the fleet. . 

• High reg[atd for mel^ 

• High -expectations from men 

■pjto^ ^ rs 2 • % 

• Ratee*s ''knowledge of the S-2E Electronic System is outstanding, and he 

is able t6 utilize this exceptional knowledge through skillful manage- 
ment of shb^*^ personnel and an innate ability to pass along what he 
knows to oiphers, t 

Ratee handles his men in an effective manner and always Igeta good re- 
suits from hlB subordinates. 

• Ratee has th^ ability to solve problems and motivate peopbe. 



C9f*P r ATT # ^ ^ M I ^ 

♦ His loyalty Was shown in the conscientious manner in which he attended 
to the problems of his men. 

# Molded crew into a competent and effective team 

tr t I l-r-J) I 

• Encourages and guides subordinates 

Er|c • 144 ^ 
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LEADERSHIP AND DIRECTING (CONT.) 

• Helped men advance in rate ^ 

• Leadership ability v 

^' ■ ■ ■ ' 

• His men are hard working, ^ 

• Performs well as a leader 

• He appears to be indifferent to the personnel administration of his 
men, especially in regard to special requests and advancement. 

• Although the tasks assigned to the personnel under his supervision are' 

always completed efficiently and in an excellent to outstanding manner, 

I. 1^ OP -/ 

his superiors feel that he is Qapable of getting more out of his men 

rif M -I 

and of putting more of himself into the Job. 

t z> -I 

• Lack of leadership 

f V- M 4.*^ J) -/ 

• Needs more forceful approach to leadership 

. ' LEADERSHIP AMP PIRECTIM6 oZio mixSbi bo, cAo^cuUng an cUmo6phtn.z ihat mafeei 
tzcm)o/ik po66lbl2,, 6ucih 06 mpwvlng rnKfUng cjondi£Loyu>\ A moMUJiz mold 
be monjoJid. ' 

• Promotes CinspireiS dr promotes) high morale 

• Contributes to high morale 

/. ^ ^ a , 

■% High shop esprit 

• High regard "for team concept 

• Promotes harmony and accord 

• His men are cheerful. 

• No disciplinary problems ^ 

LEAPERSHIP AMP PIRECTING indbxdd aJLl oi tho6z activitiu u)lvidi evil dt^ign- 
ad to (LntouMLQd ^ahofidlnatu to mnk zlizcjtiodiy and Q.UtcA.Q.ntiy . Tlu^ 
JU n.(i{^l(icX(Ld in tka my In uohJick tha indivtduaJL boJMQ (L\)aJUuvtzd fioMtu 
to, kl6 6abo^dinatz6 . 
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LEADERSHIP AND DIRECTING (CONT.) ' 

He Is tactfxil with his subordinates, 

• Tact in handling subordinates 

Tht iuvppont that an lyidLvtdacLt has iJoWi hU 6ubo/idiyiatz divisional pQA&on"* 
noZ muld bt Indoxad mJjk LEADERSHIP AMP VIVECJIHG, How^ve/t, an Indlvldu-- 
at^6 tyit2Aptuonal KoZatiomkip ntth kU pzQA6 on. 6u.p2/Uon^ pn,obably voouZd 
be -indexed by onz.o^ thz ioWmiiiig tznm: COWUCT AHV ATTtTUVEj o/i CO- 
OPERATION m nESVOHsmHESS. Thz JtappoU that an IndLvmuuxl has volth 
. othoA o/tganlzatLonat units, aiiexits hXs division^ 6 wo/ik and, tkoJizf^onz, thz^ 
Inddx tmn REPRESENTATION would be* usoA, ThoAZ cw.e tm exceptions hhoAe 
thz /iQ^atlonshlp between an Individual and his subo/idlnatu muZd not be 
Indexed as LBAVERSm? ANV VIRECTJNG. Wkzn thz statmznt has to do uoUh cCn 
Individual's coopznatlvz aXtltadz on. "mnklng vooJW tjoltk othQJU-"6upe/U- 
onjS on. subondlnatu [and not his eZccltlng coopoAotlon in.om otho/is}, use, 
th(L index t2Am COOPEmiON ANV RESPONSIVENESS. Pon. oxamplz, "He Is co- 
opQAotlvz uiUh supoJilon^ and subondlnato^s muld bz -Indexed COOPERATION 
AND RESPONSIVENESS 1. The. otheJi exazptlon Is a statement about the. men'* 
'n.ejspe.(it on nega/id ion the. Individual. Von. example, both ''his men t/wust 
him," and 'ithz men anz loyal to him," muld bz Indexed by tht teJm RECOG- 
NITION 1. 

• Works well with superiors and subordinates ' ^ 

• Cooperative with superiors and subordinates 

• Willing to help superiors and subordinates 

• Willing to help his subordinates 

A manage/i In his leadeMhlp Kole. mu6t act his pant and be, aonscloas o^ thz 
Impact OfJ kU behavlon on his* men. Von example,, "He leads by example.," 
muld be. lnde.xe.d as LEADERSHIP ANV DIRECTING U HomveA, li a nzieJizncz 
Is made, to an Individual' 6 conduct as 6eXtlng an ^xmple.. It Is as6umexl 
that ''^example.'' qaaHileA the. conduct and dozs not /icie/t to tke. IndHvldu- 
al'i leade/Lshlp abKlty 6lnc^ no ne^^eAence. Is made, to his lejxdejvshlp nole.^ 
The/icionc, the phnases "cxemplanxj behavton" on "his behavlon Is an example, 
to all" on "...to his men" muld be Indexed as CONDUCT AND ATTITIWE 2 and 
CONDUCT AND ATTITUDE 1, neipejctively. 

• Leads by setting the example 

t V p I 

• Mindful of his position as a leader | 

The. mxdjs supe/ivlslng , dJAeMlng, on managing can apply to tasks, men, on. 
both. The. IndexeA has to judge, i^hat the. e.vahxatan means. It usually can 



LEADERSHIP AND DIRECTING (CONT.) 

be cu&mzd that ^apaJwUing nzicM to omqmM mamgeAXal ^ancUoru> 
ta6k6 (MANAGEMENT FUNCTIOWS), unlUA Atatzd vK XniaJVtad othemUz, 
thz 6tatmznt fizioM to the. AupeAu^-twg, dLUzcZinQ, managing oi 
only, ikzn aie LEAPERSHIP ANP PIRECTIN6. 

• Consistent in direction of personnel 

• Directs men well 

I.T-0 I \ LfJf I 

• Finds time to direct and counsel young men 

• Skilled at managing his men 
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MANAGEMENT FUNCTIONS refer to those Job duties which are characteMstic of all 
Bianagers: leadership and directing, organization and staffing, planning-' 
controlling, and representation* Though operations inay differ from one 
organization to another, the functions of the manager are common to £^11. 
When the specific managerial function Is mentioned In the narrative text, 
use the more specific Index term under MANAGiSllENT FUNCTIONS In the 
hierarchy ♦ • • 

EXAMPLES: 

4 - 

# He Is an excellent manager and organizer who Is willing to accept any 
assignment no matter how difficult, 

m Chief has made a prime contribution to the ship through his excellent 
supervision of the Fuel Oil and Water Testing Laboratory and the Oil 
Kings* ^ 

# He is well versed in the 3-M System and always exhibits sound manage- 
ment practices. 

^ i- 3 

# iRatee's superior leadership capabilities and overall knowledge of 

management greatly contributed to this division receiving a grade of 
4.0 djuring the annual administrative inspection. 

, m BTC has an excellent working and practical knowledge of the EMS System 

. M A? -I 

but has a tendency to be lax in the administrative phas.e of the system. 

Thz mp2A\)ls>imf xtOizcttng, ok mamg^ng cm apply to toAkb ok to 

men. Thz IndoxoA hai to judgz what the, Qy^aJtixataK meawA. It u&ugily can 
be oA^umerf that 4>upeAvUtnQ kz^qm to the, ovqMjoJUL manoQ^Alal ^unctloM o/i 
ta6k6, unteM 6tatzd ok InloAJuLd othomlt^z. IjJ thz ^tatm^nt Kti2A6 to 
^ thz 6ap2Avl6lng,,(iU(i(itLng, ok managing mm only, thm a6e LEAPERSffXP 

# He has demonstrated superior performance in supervising equipment, 
maintenance, operation and repair. 

# His administrative knowleidge and ability to supervise and coordinate 
the efforts of other Instructors enabled this command to develop all 
the material required for realistic support of the E2B Aircraft, 
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ORGANIZATION AND STAFFING refer to the establishment ^of an Intentional struc- 
ture of roles through the determination and enumeration of activities re- 
quired to achieve enterprise, goals, and the manning of and keeping manned 
these roles or positions, 

EXAMPLES: 

' OKQCLYilzjoution IncXxidoM QKouping acMvAjUoJ^and Kolti, deZzgattng aiUho/Uty, 
ami coo^tdLfxcUlng auXlwn^ty K(djoutioMhvp^ . . 



1(5 an lYuilvldixal azU up a Uai&on v^Uk anoihoA o/igafUzcUloml uifuU on. 
division tAJlthin thz Navy ok an ovuO^Jidt oKgdnLzatign, thz indzx torn OR- 
GMUATJOU Af4V STAFFING would be a6ed. Howzv2A, li an IndlvldmZ u^Qj>'^jth(L 
UiiUon OK oKQanlzatioml 6tKacXuA,z that l6 aJUojody 62t up to en/iance fiU 
dlvtAlon^^ mKking Ktta£ion6hlp voWt otk^ oKganizatLonat untt^, tiiz Index 
t£Am Rt?R^ENTATWN would bz u6zd. , ' ■ ^ 

• He does an excellent joh o£ planning, organizing, an)d carrying out his 
job, ^ 

0^^ 3 ' 

• He has developed an extreiaely tightly knit division which has an 
uncommon amount of pride in its work» 

• Excellent job setting up the operation 

• His ability to assign workload in a smooth fashion is noteworthy. 

^^Ma '.^ 

• He is a highly motivated and aggressive individual with a good sense 

0^ s I f^^ I 

of organization and administrativjB ability. ^ 

g TM - I ' ^ J ^ 

• Ratee spends a very limited time in the shop but has exerted a spirit 
of independence in his First Class, the result being a well organized 

* and efficient shop. 

' /HI*/ 

• He is a professional administrator and understands the princifjles of 
delegation .w i 



ratee' s ability tb work with others, his capacity for organization 
and stimulating enthusiasm makes him a valuable asset to any unit. 



0¥S 9 ^ . 

He reorganized the work center. ^ ^ 

Set up a file on each item 
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ORGANIZATION AND STAFFING (CONT.) 

• Ability to coordinate y 

^ . * * * - • ' 

O ih S I ' . ^ 

• Coordinates work centers * % ' 

0^ S I ^ 

• Coordination of work duties 

• His initiative at times lags, and he tends *tQ undertake too many tasks 
alone rather than delegating them to subordinates. 

StaiilnQ incJUidoM thz appruiUaJL ok aA4e64me>tt OjJ poMonnoZ to 6zz li thzy 
cJiz 6UAXzd to a Kolz OK po^jution, ^and thz tKoinlng o^ pe/uonnzt iMh tho, 
cum 0^ having thm cjoaaij out thaiK koZqj^ cu dz^inzd by thz oKQanlzaZiomt 
6tnjULQJtiJJtz. P&Uodld apphjouUat oi poA^omoZ madz to doXoAmim how thz 
pzMonmZ oKz cMAyAJig oat thoAA koZ,qm and to <U4eA4 t^hzthoA ^u/ithz/t 
t/uUning jU nzdOM^OAy^'oK whoXhzA pKomotion^- OKZ m/uuintzd. 

Co/A Ml" c y s 3 

• His instructions are presented in such a manner that maximum training 

is accolaplished in the time alio ted. 

• He speiit a great deal of ejctra time with his ^men and turned out well 

0 f 9 

train'ea, well motivated men for the fleet. 

• He is/ continually researching the available training and ensuring that 

assiflfned personnel have what is needed. 

? ^ 

m Rate/e Jhas tutored division personnel in all aspects of the Navy publi- 
cations system. . 

• His men re-enlist. 




PlJUSNING-CpNT]ROLLING refers to the decisiQn-inaHng process Invalving the 'selec- 
/. tion among alt ernatives^pf. objectives policies^ and programs^; and the 
means for achieving and assuring the accomplishment of plans. Control 
which Involves the accomplishment * of plans is assured by measuring per-* 
inance against established standards or goals and correctlngy&eviations ajt 
taking action that prevents deviations from occurring, Pl^pning-control 
systems involve establishing and maintaining objectives or goals; poli- 
cies; methods, ways, or pro ceduresl'^Yules; programs; budgets; strategies; 
schedules J and feedback mechanisms such as maintaining inyentory-tra:fflc 
flow patterns, checks and balances, bookkeeping, and accounting systems^ 

EXAMPLES: 

• He dpes an excellent job of planning, organizing, and carrying out his 

' -jobJ ^ • " ■•■ ■ ^. ■ 

• His abJLlity to seek out potential problems and correct them before 
failure occurred has been extremely beneficial/ ^ 

• He is ^consistently capable of resolving jproblem areas before a critical 
situation Can develop. ' « ^ 

< 9 

• In his capacity as water chief tender he has enforced a strict and pro- 
- fessional water chemistry program. 

• He is "methodical, deliberate, and able to develop effective and effi- . 
cient procedural methods of accomplishing the division workload. 

• Rafee's resourcefulness in setting up a procedure to instruct: 150 re- 
\ » *■ 

emits daily in this very difficult operation was largely responsible 

for the efficient operation of the matches. • ^ 

• He had the ability and initiative to plan and assign work to personnel 
under his direction to ensure that the end results are of the highest 

. quality. • ♦ 

^ f ' ' ^ 

• His analysis of divisional problems, both functional and administra- 

^ ■ ' ^ ' K 

tiVB, and the execution of corrective measures h^e been very conducive 

[ . to a smooth and highly effective divfisioh. 

I • Found new ways to improve the security of the Communications building 
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PLANNING^GONTROLLING (CONT.) 

■ ^ 

: , ' # Good decision maker 

• Show§ judgment * 

• Improves plans v 

o * . 

• Has alternate plans* ; ' 7 

• Solves broblems . ^ 
^rle^'Ht^^roblems / ' » 

Corrects_difficult±es 

Because he never examined the fireroom equipment during the 10 days 
iit-port, the unsatisfactory condition of the boilers went unchecked. 

and l6 poAiomod coldly by an IndXvldixat imtojxd oi by a gAoap 04 kU 4afa- 
ojtdlyuvtUj It 6houZd fae ^^goAdzd mo fid 06 a tddinlcaZ imciion ofi thd dm- 
on^tMXloYi oi a tdchyitaaZ ^kJUUi nathm than conildoAdd to bd a cont/ioJ^JUng 
iunctlon, 'and muJLd bd iabaJidd TECHNICAL SKILLS. Von. dxampld, "Ratdd^^ 
knoiAilddgd oi thd P-3 OAjicAxiit dldotnonic ^y^tem and kU abulty to ox- 
pddAjtiouAZy conAdct thd mo6t complzx oZdcMonlc pnoblm ^ oivUtandinQ 
mold bd Inddxed PROfESSJONAL AND TECHNKAL SKILLS 3; PROFESSIONAL AND 
TECHNICAL SKILLS 3. Thd modliy^ng advoAb oatbtandinQ baJtongA to both 
ph/uuu 0^ tkU bdntdncd. ^ 

'. ' ' ' ^ ' 

\ • , • • * 

/ * • . • - 
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PRODUCTIVITY AND ACHIEVEMENT refer to the accomplishment and accomplishing of 

a desired result achieved through an individual's performance of his over-^ 
all job duties as a manager. Included here are .statements referring to 
the ongoing performance or an individual in executing his job duties as a 
manager that has led to desired results in the past or that is expected to 
produce d^sired results. ■ " , 

EXAMPLES: 

\ fr S 3 

• He carries out his mission to the best of his ability. 

■ / ' ■ ■ 
^rA 3 • / 

• Ratee's per;jformance is outstanding in ajl aspects. 

/' 

• Performance in both his primary and secondary billets has, been out- 
standing in all aspects as marked in block 19. * 

• • Always work$ to his fullest potential 

• Never allowed shop to wane 

• Handlad duties ^ outstanding manner 

' ^r il 3 

m He never leaves a job unfinished. 

• His thoroughness in performing. his duty is outstanding. 

• Outstanding in his work 

3 / - 

• He has the ability to perform in an outstanding manner. / 

/ir^3 

• No error noted\ 

• Meets responsibilities in a minimum of time 

/ r ^ a ' 

• Every assignment is performed with unusual Accuracy and effectiveness. 

• Under his supe^ision th3 shop has met and surpassed the accepted norm 
^ of productivity. . 

• RM 1 has hit his stride and surpassed even his previously high stand- 
, ards. 



/'TAX 

f Impressive discharge of duty 
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PRODUCTIVITY AND ACHIEVEMENT (OONT.) 



^ONT.) 



• Performed remarkably well 

fr A I 

• Production over the pa^t six months has been about average for "a two- 
inan station. 

* • Tested and proven und&r fire 

• Success in a difficult task 

f^0f^ r TS I S^A I 

• His technical competence and resourcefulness contributes to ships in 

f ft A I ■ * — 

tending to maintain operational readiness. . 

A ) ^ * 

• His performance has been commensurate with his" rate. 

/ , ^ 

• Capable of handling a demanding job in an efficient manner 

• He performs his job as radio supervisor in a conscientious and reliable 
manner. s 

fTAI • 

• He performs in a capable manner. 

A I 

• Assignments are completed (or completes assigned tasks). 

• Chief Is a performer. 

• Has seen these tasks through to their successful completion 

Al V 

• Carries out all assignments expediently and efficiently 

e^A I 

• Efforts bring success 

e^A I 

• Overcon^es obstacles , * - 

f He me^ts-Jiis responsibilities and quotas in a timely manner (or ahead 
of schedule).^ 

f While h6 was acting as ship's Oil King, the ship witnessed three oil 
spills while refueling. 
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PRODUCTIVITY AND ACHIEVEMENT (CONT.) 

• His extremely poor performance of his duties led directly to the ship s 
boilers not being In a state of operational readiness. 

SUtmzYvU oi JbnpiovmzYvt should fae indexed PROPUCTIl/Iiy AMP kmE\>t- 
MEN! thztj (Viz In tzAM ovomZl pQAionmdYizz. ^ ' 

• Made significant Improvements 

• Made , Improvements ♦ ' 
m Enabled him to Improve 

• Yielded benefits jT 

• Room for improvement 

• Expected to improve 

'It imponXant to makz dznXjoU^n that thz acklzvmzifvt u}cu> not a pzuoyiat 
tzzhnlcjil dZdompUMhmznt, In voUck code PROFESSIOWAL AMP TECHWICAL SKlllS 
muJLd bz u^zd. 

fit^r t T$ I 

• Ratee single-handedly tore down and rebuilt an engine in 16 hours so 
the ship would be ready for the cruise. 

UAe PR(}t?UCTII/ITy AWP ACHIEl/EMEWT an awaJid l6 glvzn to an IndLvlduual' 6 
6quad/ion, command, ok QtkzK 6uch unit. Vo\ zxampiz, "He coyit/Ubutzd 
dUizctly to thz anmal OP-El/AL am^d o{^ ' OuJUtandlnQ' glvzn tkU division 
by thz VCA Im^pzction Tzam du/Ung thzVi KzcznX, vUit to th<A Command.'' 
FROVUCTIVny and ACHIEI/EMEMT 3. RECOGNITION u)ouZd bz ix^zd li thz am/id 
had bzzn glvzn tq^^z Kotzz. 

• It is also noted that he hadM^ieen selected for the NARTU Lafcehurst 
"Sailor' of the Quarter" award because of his outstanding continuous 
record of achievement. 

/f^ c 3 Z.r7 / 

• BTC was given an achievement medal for his leadership efforts during 

the 1969 WESTPAC Cruise. 
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PROFESSIONAL AND TECHNICAL SKILLS refer to the experience, knowledge, and the 

understanding and demonstration of techniques and technical skills In a 
specified role. Includes professional, professionalism, technical 1 
knowledge, expertise, technical background, knowledge of the responsibili- 
ties of rate, understanding of job duties, technical experience. 

a 6pz(ii{^ic function ok bohavijoh JU qaaiyLf^Zod by thz above, concipt ami 
th2Az ^ an^ appKopnAjotz tznm In thz hlanxvichj, thzn thz tabzZ^^ytdA.aatlng^ 
thz 6pz(ii{^la function ok bzhavloK, 4>uch 06 MAWAGEMEWT VUhlCTlOUs, muZd be 
' a6ed. Tkz conazpt o<J pKo{^U6ZomZ and tzchnlaaZ 6kiZJU muZd bz contldzK- 
ed to bz a (luatiilzK {^ok thz ZabeZ u^zd: HoMzveA, orfien tkz abovz conczpt 
modL^A^z^ a Kz^QAzncz to a 6kUJi, abiJCity, ok qaaJU^^^cjation In a ^pzoA^ilc. 
Kolz ioK MhLch thzKz l6 no tabeZ In thz ktzAoKcky, u^z thz, tznm VROVES- 
SJOhlAL m TECHhlKAL SKILLS. SKILLS AMP ABILITIES mold only be tued {^OK 
a gznzKot Kz^zKzncz to a 6kWi and not to a 6kiiZ In a ^pzcX^lzd Koiz. 
VoK zxamplzi 

^ TS 3 ^ . 

• Extremely effective ET ^ 

• ' Professional attitude 

Cm/itp r ATT I 

^ 0 Professional behavior 

M ^ I 

• Professional administrator 

• Knowledgeable leader 

• He is skilled. 



fA0P ^ ts I 

Professional skill 

Versatile Instructor 
Knowledgeable Radioman 
Ability to teach 
EXAMPLES: / : 

Ratee has consistently demonstrated his outstanding professional 
qualifications. 

The ratee has an outstanding knowledge of all Electrical Drone Systems 
in the squadron aircraft* 
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PROFESSIONAL AND TECHNICAL SKILLS (CONT,) 

m BTC has an excellent working and practical knowledge of the PMS system, 

>•! ^ I 

but has a tendency to be lax in the administrative ph^e of the system*. 

• These accomplishments were achieved through high professionalism, en- 
thusiasm, and superior decision-making ability. 

• His expertise is widely acknowledged. 

fMlP r TS / 

m As Airborne Communications Supervisor, ratee*s professional compe- 
tence has contri^uted^ta flight operation, excellent mission perform- 
.ance, and' praise for his division. ^ 

• Ratee single-handedly tore down and rebuilt an engine in 16 hours so 
the ship would be ready for the cruise. 

^ A \ /^vf r -rs I . 

• His performance reflects competence and prof essionalism. 
• - <* 

fii$,p rrS I , M r ^ ^ 

• He is well versed in the 3-M System and always exhibits sound manage- 
ment pract ices . 

^^•^ ^ rs i s f ^ ^, ^ 

• His technical competence and resourcefulness contribute to ship^ In 

tending to maintain operational re^iness. 

f40/ T rs ^ i 

• ProMssional knowledge 

• He knows what is expected of him. 

• Chief wasVelieved of his duties as the ship's Oil King after serving 
in the capacity for approximately two months. He was removed from this 
billet because of his lack of professional knowledge and technical 
know-how in the art of refueling. 

Wfeen an Individual jU In thz pfiodOM^ 0|J acq0Ung now knou)tzdgz ofi a new) 
6fult, it (joouZd be an ImiiavUon IvU motlvcuUon and be laboXzd cus EN- 
VUmCE ANV MOTIVATION. Thz IndzxdA should not a6e PROBSSIONAL ANV TECH- 
NICAL SKILLS unt(^6 tkd individual hcu^ {^inAJ>h^d thz coime and, thvizioKz, 
' aJUnadtj ha6 tha knoi/^zdqz on. 6fUil. 
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PROFESSIONAL AND TECHNI(ki SKILLS (CONT.) ' ' 

# He is a very^ knowledgeable technician who attempts to keep constant^ 
. ^breast of current changes by devoting his -off duty time to the study 

of technical and nontechnical material through evening college courses 

# Keeps well informed on supply procedures 

# Keet)s up with changes in publications 
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RECOGNITION refers to the acknowledgment made by others of an 'individual's 

standing in relatit)n to his worth ojr value to the Navy, an organizational 
unit, and /or the men with whom he works. 

EXAMPLES : * 

• He possesses the character, personality, and desire that commands the 
highest respect and admiration from his associates. 



• He was given an excellent overall grade. 

• This action has generated |:h# highest degree of trust and confidence 
with his superiors. 

• He Is well liked and highly respected by Juniors and seniors alike, and 
his easy-going manner and pleasant congeniality make hiiaF at home In any 
surroundings. 

• Ratee is highly regarded by all of his men. 

• He has their 'full respect and gets "it. . 

• Inspires respect ^ 

tts X 2. ictf c -a 

• Expertise widely acknowledged and respected by others 

• Gains the genuine respect - * ^ 

• He expresses himself clearly and logically and his views are respected 
by those with whom he works. 

• He readily obtains the confidence of all wko come in contact with ^^^^ 

• Men trust him/ loyal to him 

• Praised/popular 

• Justified others' confidence in him * 
m Commands respect and gets it 

m His advice is sought by others because of his technical skill. 
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RECOGNinON (CONT.) 

y • His concern for his men has won the respect of his subordinates. 

• Merits the respect 

• I would welcome him in my crdw anytime. 

• Generates respect/respected by 

^ • ' * 

IX an Individual U con&ideAod to bz an (U6Qt on, cJindlt to tht 6eAvl<iz, 
then iiAt HECOGNJTION. HomveA, li one oi kU tnjcuUUs on. 6kAJU6 U an cu^ejt 
to tht Navy on In poA^onmling a tuk, thm 04 e "044 et" a6 a qwxU^lqA ion. 
tht tAoit on 6klil. Von oxamptt, "H^ coopoAxsuUvz natoAz jU a gneat a^^zt 
to tkt Navy." COOPERATION ANV RESPONSIVENESS 2. Statmznt6 about nt- 
jlzctlng cJuudU upon tht Navy muJtd :fae Indaxtd 06 REPRESENTATION.^ 

• Ratee has been an exceptionally fine asset to this command. 

3 

• One of the most outstanding men in the^ Navy today 

40^ 3 

• He is an outstanding military man. 

• He is valtlable to any command. * 

• He is a valuable asset to the service. \ 



He is a great value to the U.S. Navy. 



DESCOL and the Navy. 

• Able to handle varied jobs and hence is a very valjj&ble Chief 

• Repiorted in a commendatory way \ 

' /^40^ r*T'S 3 

• Ratee has proven to be a definite asset with his outstanding profes- • 

sional and Instructional ability in a relatively short time. 

• He is an asset to the ship and the naval service. 

• He is a credit to the Navy. 

He is a credit to the squadron. 
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RECOGNITION (CONT.) * ' ' 

• He is an asset to the command of the Navy. 

• He is an asset to hjgh morale. ^ - 

RECOGNITION indhidz^ m oHIqajxI oimaA on, puYUAlmzYvt, oK ifie nzQpmmzndd- 
tion ion, 6{iQk on mtvid on pzywUUy, iA}kLch mold 6e oi^x^Kied a wuQht o^ 
(UjthoA 3 on -2, nupzdtlvQjiy. U6e RECOGNITION only onct no moJXoA how 
many moKd^ an Individual Kzczivzd li thzy anz all montlomd In om ^tatz- 
mznt. ¥on zxamplz, "He won ilvt awoAd^ ion ocuUtandlna pz/Liomancz Icut 
ymjt," would be indzxzd 06 RECOGNITION 3, PROPUCTZl/TTy AMP ACHlEl/EMENT 3. 

' ll6z PROVliCTlVlTV ANP ACHIEl/EMENT 1^ an omAd aJ>, glvzn to an Individual' 6 
6qixadn.on, comand, on otkoA 6uch unit. Von example, "He dowtJilbutzd 
dUzctly to thd annual OP-El/AL auooKd oi 'OuX:6tandlng' glvzn thl6 division 
hy thz PCA Jn^pzctlon Tom duAlng thoMi nacznt vli^il to tkU Xormand." 
' PROPUCTIl/ITV ANP ACHIEVEMENT 3. RECOGNITION would fae a6ed 4!^ th^ omAd 
had hzzn glvzn to thz individual. 

• It is also noted that he has been selected for the NARTU Lakehurst 

* MC 3 

."Sailor of the Quarter" award because of his outstanding continuous 
record of achievement. 

• BTC was given an achievement medal for his leadership efforts during 



the 1969 WESTPAC Cruise- 



• Awarded the Navy Commendation Medal for meritorious service while 
attached to Fighter Squadron ONE FIVE ONE embarked in USS CORAL SEA 
(CVA-43) during combat operations from 10 October 1968 to 30 March 1969 

• He was awarded an oral admonition. 4r 

• Relieved of duties 

« 

RECOGNITION also l6 mbodltd In 6tatmwt6 nzcomxmndlng an Individual on 
acknowledging that an Individual Is ^ultQjd ion a nank InCAQA^z, 'advancz- 
mznJt, pnomotion, on addzd n'eApon^lbltiXlzs . 

• Ratee's performance in the past was responsible for his selection from 
every First Class and Chief Petty Officer in this coimnand as Command 
Career Counselor. 

• . He would be most valuable in an instructor billet assigrxment. 
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RECOGNITION (CONT.) ^ 

• He is highly recontm^ded for advancement. 

• He is highly reconnended advancement and retention in the Naval 
VReserve. . ^ ^ , 

• Strongly recommended for promotion • ^ 

St /I 3 /i^c / 

• Outstanding potential for added responsibility 

S^A J A^c! f ' § 

• He is qualified to assume -t;he greater responsibility of the next higljer 

pay grade. , ^ r ' 

r ' 

• Ratee is fully qualified for advancement in rate. 

^ fAor T rs X ^ A€C I 

• Ratee is very knowledgeable in the supply system and is recommended for 

E-8. 

1» • 

• feminently well qualified for advancement 

4 i /r^c / 

Capable .of assuming more responsibility 

• Has passed the exam, but was not advanced 
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REPRESENTATION refers to the creation of an image of^ an organization to the ex- 
terrial or internal environment. An organizational unit^ image affects 
its working relationship with other organizational units within the U.S. 
Navy an<f/or with the community outside vof the organizational structure. 

Some management experts do not consider REPRESENTATION to be a separate 
managerial function. However, in this dictionary itJLs considered to be 
a separate function because of the importance given to relating to the 
civilian community by naval personnel. ' 

EXAMPLES: * * 

• He is an outstanding representative of the Navy in all respects. 

• Ratee has been very successful in improving his Branch Stationjs rela- 
4 tions vith local high schools and youth organiz^ations. 

. • The appearance of this building is a fine example and reflection of 
the Navy to the^ visi-ting civilians. f 

• He is an active Navy promoter and is the first to step forward to edu- 
cate subordinates on NaVy policy, procedure, or benefits. 

• He/takes an active part in church and civic programs, helping to uphola 
the Navy image in the community. m 

• Active in extracurricular activities 

• Takes part in extracurricular activities 

• Public relations for his branch 

• Status of his branch or job area 



The. fuippoftX that an individual koA mWi othoA oKQanlzatioml anltb alio 
anhancu kin dA.vi{,ion'6 woKk, and 'thz indzx. tdJm REPRESENTmOhl maid be 
a&ed Thz nappofvt that an individual hoA Mith \iL& dLvUloml pe/uonml 
probably midd le, ind&xzd by extheA LEM)ERSHJP AWO PIRECTING o/t COOPERA- 
TION ANP RESmSWENESS. ^ ■ \ 

H an -individual 6eXA up a Uaiion uilth anotheA organizational unU on. 
division Mithin th& Navy on. an ouXAldz onganlzatlon, th& inde.x. tern ORGANl 
ZATION ANP STAFFING muld be. oied. HoweueA, U an Individual, u^eA the. 
tiau Uon on onganlzational itmicXuAe. that 16 alxzady AeJ: up ta znhancz hU 
division' 6 mnklng nelatlonAhlp with otheA onganlzaXlonal unlti, thz -tndex 
toM} REPRESENTATION would be. u^ed. 
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'^I^EPRESENTATION CCONT,) 



A i>fyxXmzKt about an IndisJidual ftz^tejcZing cJiddjut upon thz hiavy twuZd' be 
labeZzd a& REPRESEWTAnON. 

• His achievements, loyalty, and devotion to duty are in keeping with the 
highest traditions of the Armed Forces and reflect great credit upon 
himself and %he United -States Navy. ^ 

• Reflects credit on the Navy 

/ / - ; V , 

• Brings credit to the Navy 

H6C I ^ '' , 

• Is a credit to the Navy 
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SKILLS AND ABILITIES are those characteristics and qualities that influence a 
person* s performance in the attainment of goals and results. If the type 
•Df skill and/or ability is specified, and there is an 'appropriate, term in 
the hierarchy, use the more- specif ic term. 

* 

IeXAMPLES: ' - . • 

• There is nothing that can arise in his present position or next assign- 
ment that ratee cannot cdpewith. 

S r/t 3 . - 

• This man is extremely competent. 

^^P i ^ 

• He is able to direct the efforts o^ Line Personnel in an efficient and' 

effective manner; this is reflected 'in the ratee by a multiple of ex- 

5 ^ 3 / 

ceptional qualities, a ■ . I 

• > Above average in all areas, he is fully qualified to assume the greater 

respottsibility of the next higher rank, 

• - . • ■ , • 

• He daily demonstrates all of the highly desirable traits of a Chief 
Petty Officer. ' 

S T 4 2 ^ ^t4 2. 

r • Ratee is very meticulous and thorough. 

• Capable of handling any situation that may arise 

• Is able to handle varied jobs and hence is a very valuable Chief 

y.v- ^ / £#/<p T ^T-r / 

• His iiatural abilities and responsible approach to recruiting have en- 

abled the tia^e^to outperform his contemporaries. ^ 

i ■ ' ' 
£tM -/ 

• With more time and conscientious effort, he should realize a greater 

S T-A -I » V;- M 

potenMal. , ' ' J 

SfCIlLS ANI? ABILITIES l6 a catahaJU tOun ion, tko^z conczptA thxU 6tand 
alonz and do not quaJUiy otheA dJjn^zn6lon6 which hcuoz thifJji om indzxlnq 
tznm In thz kUAOAchy. Von, zxmplz, "a ^kAMial pzuon" muM fae. indexed 
04 SKILLS ANP ABILITIES, bat "mayia.g2Alal ^kJUU" moM bz Zabzlzd \Mlth thz 
index tznm mAGEMEt^ FUNCTIONS. 

^ Extremely effective ET 

5 r /I 3 

• Exceptionally well qualified 
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SKILLS AND ^ABILITIES (CONT,) 
9 Outstanding individual 

• Has tnany skills v //^v^ 

M 0P i V 

• Well-qualified Supervisor 

• Prof icietit manager 

$ 4 / . 

• Proficient individual 

- • ■ • 

S T /I I ^ 

• Proficient Chief Petty Officer 

• Able leader ^ - . 

SrA i « V ' 

• Able person _ * 

» ^ - ••.«-.- 

• Skilled technician , 

Al ^ I , 

• Skilled manager ' ^ ^ , 

• Skilled Radioman . 

• Skilled Instructor ' . 

• Experienced Instructor 

Jka ^oUou)lng t(>Jm muM be labeled SKILLS AWP ABILITIES 1^ thzy voM.t not 
quattiylng anotkeA concept ion, (jokLch thcAt l& an Indox tojurn nd^iou/ic^iul, 
potential, qmtUiu, ^ftUls, tKaid, attzntlon to d<UcuZ, mieitnt, 
aiitdtivt, pnoilcLziU, atcmatii,, m^tlcuJioiU, thorough, mojthodlcaZ, com- 
p^tznt, capablz^ and tatiMtiud. 
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